Server that can read data out of InSync H5M files.
For building/running from source:
gradle >= 5
java jdk (or openjdk) >= 11
or
docker
For running service from RPM installer:
CentOS >= 7 java jdk (or openjdk) >= 11
gradle build will build (fat) jars/tars.
gradle buildRpm will build rpm.
Replace with one of above.
docker run --rm -v "$PWD":/home/gradle/project -w /home/gradle/project gradle <task>
Example:
docker run --rm -v "$PWD":/home/gradle/project -w /home/gradle/project gradle gradle buildRpm
This will build the rpm using docker.
The jar can be ran like:
java -Djava.library.path=./lib -jar ./build/libs/insync-measurement-reader-<version>-<release>.jar <options>
To build image:
docker build . -t <image name>
To run image:
Please check docker documentation for port and volume mappings.
The application can also be installed as a systemctl service with the RPM. Install the rpm. This will create a new
file /etc/insync-measurement-reader/insync-measurement-reader.properties.example. It will create a no-login user
called insync:apache if it does not exist yet, and the service will operate under that user. The service is fault tolerant
and will restart in case of a crash.
Copy this file to:
/etc/insync-measurement-reader/insync-measurement-reader.properties and edit the properties to your wishes.
Start the service with:
systemctl start insync-measurement-reader
Stop the service with:
systemctl stop insync-measurement-reader
Restart the service with:
systemctl restart insync-measurement-reader
Logs are placed in:
/var/log/insync-measurement-reader.log
and
/var/log/insync-measurement-reader.err
Options can be passed to the application using either a .properties file, or by passing as command line arguments.
| Name | Shortname | DefaultValue | Takes value | Description |
|---|---|---|---|---|
| config | c | true | Path to properties file to load options from. These will be overwritten by CLI options. | |
| directory | d | ./ | true | Directory from which h5m files should be loaded from. Defaults to current working directory. |
| port | p | 8080 | true | The ports on which the application listens for requests |
| help | h | false | Shows help screen. | |
| hdfHandleExpiryTime | PT5M | true | Maximum time after last access when HDF should be released. Uses notation used by Duration.parse | |
| hdfReadChunkSize | 3000000 | true | Amount of rows loaded when loading a chunk from a HDF. | |
| logLevel | INFO | true | Level of logging, should be either ALL, DEBUG, INFO, WARN, ERROR, FATAL, OFF, or TRACE. | |
| logFormat | %d{yyyy-MM-dd HH:mm:ss} %-5p [%t] %c{1}:%L - %m%n | true | Format pattern of log entries. | |
| threadPoolSize | 4 | true | Amount of threads to use to read from H5M's. |
These options can be passed like:
java -Djava.library.path=./lib -jar ./build/libs/insync-measurement-reader-<version>-<release>.jar -d ./some/directory --config=/path/to.config
Requests can be done through HTTP requests or WebSockets. JSON Format for both is same.
HTTP requests can be send to /, WebSocket requests to /ws
To request metadata of H5M:
{
"action": "metadata",
"params": {
"file": "/105012012_TT_19200Hz.h5m"
}
}
| Parameter | Type | Required | Description |
|---|---|---|---|
| file | String | true | File to load data from. |
Will respond with object of following structure:
{
"code": 200,
"data": {
"metadata": {
"id": 96,
"name": "/",
"type": "group",
"hdfPath": [],
"metadata": {
"applicationVersion": "0.0.21",
"libraryName": "pymarin",
"notes": "not specified",
"hdf5Version": "1.8.15",
"documentation": "https://mods.marin.nl/display/H5M",
"name": "H5M",
"description": "HDF5 MARIN Datasets File",
"userName": "not specified",
"libraryVersion": "7.13.4",
"version": "0.1",
"applicationName": "SHARK",
"dateTimeOfCreation": "2018-05-04T09:46:51.409000"
},
"children": [
{
"id": 1808,
"name": "309120105012___001_002_19205.mms",
"type": "group",
"hdfPath": [
0
],
"metadata": {
"notes": "",
"programNo": "1",
"testNo": "105012012",
"waterDensityFactor": "not specified",
"dataScale": "50.0",
"modelScale": "50.0",
"stepSize": "3.6818894168672874E-4",
"description": "not specified",
"source": "OB",
"type": "General",
"projectSubNo": "not specified",
"projectNo": "30912.0",
"measurementNo": "105012012",
"categoryNo": "1",
"experimentNo": "105012012",
"dateTimeRecordingStart": "2018-05-04T13:52:40.702000"
},
"children": [
{
"id": 313434608,
"name": "F.PAN_A01",
"type": "measurement",
"hdfPath": [
0,
0
],
"metadata": {
"bases": "8224",
"unit": "N",
"baseNames": "Time",
"channelNo": "29",
"signalType": "not specified",
"description": "not specified"
},
"start": 25076.802734375,
"end": 4268.16552734375,
"min": -101722.1796875,
"max": 156737.046875,
"unit": "N",
"frequency": 2715.9969035008344,
"length": 39178192,
"bits": 32,
"scaledFrequency": 19204.998281471053
}
]
}
]
}
}
}
This represents the structure of a H5M file (without the actual measurement data). For more information regarding the metadata fields, please check the H5M documentation. An H5M is structured of groups and measurements. The top level object is a group. A group can contain other groups or measurements.
There are certain fields added that are specific to this application:
| Field name | Type | Description |
|---|---|---|
| type | "group" or "measurement" | Type of object. |
| hdfPath | Array | Represents the lookup path of the containing object inside the HDF structure. This path: [0, 1], means the second group (1) of the first group (0) inside the H5M. |
| name | String | Name of object. |
| start | Double | First value of measurement. |
| end | Double | Last value of measurement. |
| min | Double | Minimum value of measurement. |
| max | Double | Maximum value of measurement. |
| unit | String | The unit (like m/s for meter per second) of the measurement. |
| frequency | Double | The amount of rows per second. |
| length | Long | Amount of rows inside measurement. |
| bits | Integer | How many bits are used to store the measurement value. |
| scaledFrequency | Double | The frequency multiplied by the square root of the model scale of the H5M. |
| The response for metadata requests are cached in memory. So first requests to a new measurement file will be slow. Cache is cleared on application restart. |
To request data from measurements:
{
"action": "data",
"params": {
"file": "31493_03DWB_09_015_002_01.h5m",
"measurements": ["M.P15", "M.P2", "M.P3"],
"offset": 10,
"limit": 20,
"rows": 100,
"unit": "SECONDS"
}
}
| Parameter | Type | Required | Description |
|---|---|---|---|
| file | String | true | File to load data from. |
| measurements | Array | true | Measurements to get data from. |
| offset | Double | false | Offset in seconds. |
| limit | Double | false | Amount of time to get from offset. If omitted until end of file |
| rows | Integer | false | Maximum amount of rows to retrieve. Defaults to 1. |
| unit | String "MILLI" or "SECONDS" | false | Unit of passed and returned time values. Defaults to "SECONDS". |
| Will response with an object of following structure: |
{
"code": 200,
"data": {
"file": "30290_04SMB_02_009_001_03.h5m",
"measurements": {
"M.P15": [
[
-10.0,
"NaN"
],
[
10.000090792988974,
8492.865980762574
],
[
10.028240448983478,
4161.572667106468
],
[
10.056618656661414,
1318.094515885054
],
[
33.0,
"NaN"
]
]
}
}
}
Each point is represented as an array, in which the first value is the time and the second value the actual measurement value.
If a negative offset is used, the first value of the measurements will be a time/value pair of [<offset>, "NaN"]. This indicates that the requested time was before the start.
If the offset + limit is larger than the last time value in the measurement, the last time/value pair will be [<offset + limit>, "NaN"] to indicate that the end of the measurement was reached.
Content type
Image
Digest
Size
238.3 MB
Last updated
over 5 years ago
docker pull noterik/insync-measurement-reader