Sign inSign up

weevenetwork/outlier-detector

By weevenetwork

Updated over 3 years ago

Image
0

461

weevenetwork/outlier-detector repository overview

Outlier Detector

NameOutlier Detector
Versionv2.0.0
GitHubweeve-modules/outlier-detector
authorsJakub Grzelak

Description

Outlier Detector is a processing module responsible for data sanitization and anomaly detection of data passing through weeve data services. Outlier Detector checks if received data are within constraints associated with some maximum and minimum acceptance value or change rate. If the data are outside of the constraints, the module can either remove them, smooth them or keep them. The filter parameters are optional. If they are not provided, the module will not perform any filtering.

Environment Variables

Module Specific

The following module configurations can be provided in a data service designer section on weeve platform:

Environment VariablestypeDescription
UPPER_THRESHOLDfloatAny data points exceeding this threshold should be considered outliers.
LOWER_THRESHOLDfloatAny data points below this threshold should be considered outliers.
RATE_OF_CHANGE_UPPER_THRESHOLDfloatAny data point where the rate of change (per second) is greater than this value should be considered an outlier.
RATE_OF_CHANGE_LOWER_THRESHOLDfloatAny data point where the rate of change (per second) is less than this value should be considered an outlier.
OUTLIER_POLICYstringWhat to do with outliers: keep, remove, smooth
INPUT_DATA_LABELstringThe input label on which to detect outliers.
INPUT_TIME_LABELstringJSON key for the timestamp. The timestamp should in epoch format (int).
Set by the weeve Agent on the edge-node

Other features required for establishing the inter-container communication between modules in a data service are set by weeve agent.

Environment VariablestypeDescription
MODULE_NAMEstringName of the module
MODULE_TYPEstringType of the module (Input, Processing, Output)
LOG_LEVELstringLevel of logging (DEBUG, INFO, WARNING, ERROR, CRITICAL)
EGRESS_URLSstringHTTP ReST endpoints for the next modules
INGRESS_HOSTstringHost to which data will be received
INGRESS_PORTstringPort to which data will be received

Module Testing

To test module navigate to test directory. In test/assets edit both .json file to provide input for the module and expected output. During a test, data received from the listeners are compared against expected output data. You can run tests with make run_test.

Dependencies

bottle
requests

Input

Input to this module is:

  • JSON body single object, example:
{
    "temperature": 15,
    "timestamp": 1234567890
}

Output

Output of this module is as follows and depending on cleaning settings.

  • JSON body single object, example:
{
    "temperature": 15,
    "timestamp": 1234567890
}

Tag summary

Content type

Image

Digest

sha256:7854f829b

Size

48.7 MB

Last updated

over 3 years ago

docker pull weevenetwork/outlier-detector:v2.0.1