This NodeJS project is part of the KomMonitor spatial data infrastructure. As a processing component it provides a job-based computation of indicators based on georesources and other indicators within custom scripts.
This Processig Engine API aims to provide REST API functions to trigger and query process execution via so called jobs that shall be executed asynchronously. Hereby, it consumes various data (i.e. custom scripts including process parameters, georesources, other indicators and spatial units) from the KomMonitor Data Management component to compute target indicators for target spatial units and target timestamps. In general, there are two types of processing operations:
The described REST operations are specified using Swagger/OpenAPI v3. The corresponding openapi.yaml containing the REST API specification is located at api/openapi.yaml. To inspect the REST API you may use the online Swagger Editor or, having access to a running instance of the KomMonitor Processing Engine REST API simply navigate to <pathToDeyployedInstance>/docs, e.g. localhost:8086/docs.
KomMonitor Processing Engine requires
Only contains subset of whole KomMonitor stack to focus on the config parameters of this component. Only contains subset of whole KomMonitor stack to focus on the config parameters of this component. See separate KomMonitor docker repository for full information on launching all KomMonitor components via docker.
version: '2.1'
networks:
kommonitor:
name: kommonitor
services:
# redis databse required for processing engine to store indicator computation job status
redis:
image: redis:alpine
container_name: redis
#restart: unless-stopped
networks:
- kommonitor
# for special scenarios multiple processing engines could be deployed - i.e. in combination with individual processing schedulers
kommonitor-processing-engine: # perfoms script-based computation of indicators based on other (base-)indicators and/or georesources for target spatial units
image: 'kommonitor/processing-engine'
container_name: kommonitor-processing-engine
#restart: unless-stopped
ports:
- "8086:8086"
networks:
- kommonitor
volumes:
- processing_jobstatus:/code/tmp # persist tmp computation status files on disk
depends_on:
- redis
environment:
- REDIS_HOST=redis # use docker name if possible; else IP
- REDIS_PORT=6379 # running redis port
- KOMMONITOR_DATA_MANAGEMENT_URL=http://kommonitor-data-management:8085/management # URL to Data Management service; use docker name and port if possible
- GEOMETRY_SIMPLIFICATION_PARAMETER_NAME=simplifyGeometries # paramter to query geometries from Data Management component
- GEOMETRY_SIMPLIFICATION_PARAMETER_VALUE=original # values are ["original", "weak", "medium", "strong"] from weak to strong the geometries are more simplified (reducing size)
- FEATURE_ID_PROPERTY_NAME=ID # KomMonitor wide setting, which property contains feature ID values - best not be changed
- FEATURE_NAME_PROPERTY_NAME=NAME # KomMonitor wide setting, which property contains feature NAME values - best not be changed
- OPEN_ROUTE_SERVICE_URL=https://ors5.fbg-hsbo.de # URL to Open Route Service instance (currently version 5 is supported)
- DISABLE_LOGS=false # optionally diable any console log
- MAX_NUMBER_OF_TARGET_DATES_PER_PUT_REQUEST=45 # setting to split up computed indicator results import/update requests; each request has the specified maximum number of indicator timestamps
- ENCRYPTION_ENABLED=false # enable/disable encrypted data retrieval from Data Management service
- ENCRYPTION_PASSWORD=password # shared secret for data encryption - must be set equally within all supporting components
- ENCRYPTION_IV_LENGTH_BYTE=16 # length of random initialization vector for encryption algorithm - must be set equally within all supporting components
- KEYCLOAK_ENABLED=true # enable/disable role-based data access using Keycloak
- KEYCLOAK_REALM=kommonitor # Keycloak realm name
- KEYCLOAK_AUTH_SERVER_URL=https://keycloak.fbg-hsbo.de/auth/ # Keycloak URL ending with "/auth/"
- KEYCLOAK_RESOURCE=kommonitor-processing-engine # Keycloak client/resource name
- KEYCLOAK_CLIENT_SECRET=keycloak-secret # keycloak client secret using access type confidential
- KOMMONITOR_ADMIN_ROLENAME=kommonitor-creator # name of kommonitor admin role within keycloak - default is 'kommonitor-creator'
- KEYCLOAK_ADMIN_RIGHTS_USER_NAME=processor # Keycloak internal user name within kommonitor-realm that has administrator role associated in order to grant rigths to fetch all data
- KEYCLOAK_ADMIN_RIGHTS_USER_PASSWORD=processor # Keycloak internal user password within kommonitor-realm that has administrator role associated in order to grant rigths to fetch all data
# database container; must use PostGIS database
# database is not required to run in docker - will be configured in Data Management component
kommonitor-db:
image: mdillon/postgis
container_name: kommonitor-db
#restart: unless-stopped
ports:
- 5432:5432
environment:
- POSTGRES_USER=kommonitor # database user (will be created on startup if not exists) - same settings in data management service
- POSTGRES_PASSWORD=kommonitor # database password (will be created on startup if not exists) - same settings in data management service
- POSTGRES_DB=kommonitor_data # database name (will be created on startup if not exists) - same settings in data management service
volumes:
- postgres_data:/var/lib/postgresql/data # persist database data on disk (crucial for compose down calls to let data survive)
networks:
- kommonitor
# Data Management component encapsulating the database access and management as REST service
kommonitor-data-management:
image: kommonitor/data-management
container_name: kommonitor-data-management
#restart: unless-stopped
depends_on:
- kommonitor-db # only if database runs as docker container as well
ports:
- "8085:8085"
networks:
- kommonitor
links:
- kommonitor-db
environment:
# omitted here for brevity
volumes:
processing_jobstatus:
postgres_data:
| Name | Organization | |
|---|---|---|
| Christian Danowski-Buhren | Bochum University of Applied Sciences | [email protected] |
| Andreas Wytzisk | Bochum University of Applied Sciences | [email protected] |
Content type
Image
Digest
sha256:dae8c93a8…
Size
97.5 MB
Last updated
12 months ago
docker pull kommonitor/processing-engine