jbordeau/warp10 repository overview

⁠Warp 10™ Docker image

⁠Quick reference

• Where to get help:

  • Website: https://warp10.io⁠
  • Google group: https://groups.google.com/group/warp10-users⁠
  • Warp 10 community slack: https://lounge.warp10.io/⁠
  • Stack Overflow: https://stackoverflow.com/questions/tagged/warp10⁠

• Warp 10 Platform repository: https://github.com/senx/warp10-platform⁠

• Warp 10 Docker repository: https://github.com/senx/warp10-docker⁠

• Maintained by: SenX⁠

⁠What is Warp 10™

The Warp 10 Platform is designed to collect, store and manipulate sensor data. Sensor data are ingested as sequences of measurements (also called time series). The Warp 10 Platform offers the possibility for each measurement to also have spatial metadata specifying the geographic coordinates and/or the elevation of the sensor at the time of the reading. Those augmented measurements form what we call Geo Time Series™ (GTS).

The easiest way to set up the Warp 10™ platform is to use Docker⁠. Officials builds are available on Docker Hub⁠ containing:

• The Warp 10™ platform for storing and analyzing Geo Time Series™
• WarpStudio: a web application aiming to allow users to interact with the platform
• Sensision: a service for monitoring Warp 10™ platform metrics

⁠Start a Warp 10™ instance

Start your image binding the external ports 8080 for Warp 10™ and 8081 for WarpStudio:

docker run -d -p 8080:8080 -p 8081:8081 warp10io/warp10:tag

... where tag is the tag specifying the Warp 10™ version you want. See the tab above for relevant tags.

⁠Mapping volume for persistence

Docker containers are easy to delete. If you delete your container instance, you will lose the Warp 10™ storage and configuration. You may want to add a volume mapping to the containers /data folder.

docker run -d -p 8080:8080 -p 8081:8081 --volume=/var/warp10:/data warp10io/warp10:tag

In this example, you bind the container internal data folder /data to your local folder /var/warp10. You must use the same --volume option in all your other docker commands on Warp 10™ image.

⁠Working in memory

You can add -e IN_MEMORY=true to pop an in-memory Warp 10™ instance. By default, it will retain all last 48 hours. This is configurable.

docker run -d -p 8080:8080 -p 8081:8081 -e IN_MEMORY=true warp10io/warp10:tag

⁠Setting JVM heap size

You can use environment variable to set the JVM heap size:

• Initial heap size (Xms) : WARP10_HEAP
• Maximum heap size (Xmx) : WARP10_HEAP_MAX

The default configuration is WARP10_HEAP=1g and WARP10_HEAP_MAX=1g

docker run -d -p 8080:8080 -p 8081:8081 -e WARP10_HEAP=8g -e WARP10_HEAP_MAX=8g warp10io/warp10:tag

⁠Continuous Integration

A 'Continuous Integration' version is available on Docker Hub with the 'ci' suffix.

This version embeds a pair of READ/WRITE tokens named respectively 'readTokenCI', 'writeTokenCI'.

Examples:

[ 'readTokenCI' '~.*' {} NOW -1 ] FETCH // Retrieve the last point for all GTS

curl -v -H 'X-Warp10-Token: writeTokenCI' --data-binary "1// test{} 42" 'http://127.0.0.1:8080/api/v0/update'

⁠Getting Tokens

The Warp 10™ platform is built with a robust security model that allows you to have a tight control of who has the right to write and/or read data. The model is structured around the concepts⁠ of data producer, data owner and application, and WRITE and READ tokens.

The 'ci' version embeds a pair of pre-generated READ/WRITE tokens named respectively 'READ', 'WRITE'. These tokens are in the predictible-tokens-for-ci/ci.tokens file.

Otherwise, for the purposes of this setup, you need to generate write and read tokens for a test application for a test user that is both the producer and the owner of the data. In order to interact with the user/token/application system, you need interactive access to Warp 10's TokenGen⁠ component.

First, get the container identifier for your running Warp 10™ image:

$ docker ps
CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                              NAMES
dc6e541e79d8        warp10io/warp10:1.2.9   "/bin/sh -c ${WARP10_"   55 seconds ago      Up 55 seconds       0.0.0.0:8080-8081->8080-8081/tcp   hopeful_einstein

Then run docker exec to run Worf on that container id:

docker exec -u warp10 -it dc6e541e79d8 warp10-standalone.sh worf appName ttl(ms)

Where appName is the chosen application name and ttl(ms) the time-of-live of the token in milliseconds.

For example, generate a pair of tokens for an application called test, with one year of livespan:

$ docker exec -u warp10 -it dc6e541e79d8 warp10-standalone.sh worf test 31536000000
default options loaded from file:/opt/warp10/etc/conf.d/.00-secrets.conf.worf
{"read":{"token":"1Uol41cpikTrY5IGUgtwHG4kZ0puh5clethlBuq2Qjs5kWaRhvQOsHKHXsnpH5.lU7GePUIZowFTblA5lkeuDeqFZGgzrmVp1RTWghrA.f5ahLbUVO0S2.","tokenIdent":"ef8cd2a9e3e15fd9","ttl":31536000000,"application":"test","owner":"d7d310cf-254e-4065-87ae-47e83a050ab3","producer":"d7d310cf-254e-4065-87ae-47e83a050ab3"},"write":{"token":"yCY6J7jKJTWgQVrNJpsgPav7ubqiZIlx0jtDibNYO5cJNzq8EziSGOszoXGmFgXFnXbI_Yq3WXg53ry4qXkWU4vkjK9tmE3cWccPbzWvo9c","tokenIdent":"cc3a63e7b7d5ca1b","ttl":31536000000,"application":"test","owner":"d7d310cf-254e-4065-87ae-47e83a050ab3","producer":"d7d310cf-254e-4065-87ae-47e83a050ab3"}}

⁠Testing the container

To test the running container, push a single GTS containing one data in the platform using your WRITE token.

curl -v -H 'X-Warp10-Token: WRITE_TOKEN' --data-binary "1// test{} 42" 'http://127.0.0.1:8080/api/v0/update'

You should receive an HTTP 200.

Get this data using your READ tokens.

curl -v --data-binary "[ 'READ_TOKEN' 'test' {} NOW -1 ] FETCH" 'http://127.0.0.1:8080/api/v0/exec'

You should receive an HTTP 200 OK with your data point in JSON format.

A full getting started⁠ is available to guide your first steps into Warp 10™.

⁠Using WarpStudio

Warp 10's WarpStudio⁠ is a web application aiming to allow users to interact with the platform in a user-friendly way, offering an alternative to command-line interaction.

⁠Build the image

If you want to build your own Warp 10 image, clone the Warp 10™ docker repository:

git clone https://github.com/senx/warp10-docker.git

Execute docker build inside your local copy:

cd warp10-docker
docker build -t myrepo/warp10:x.y.z .

In this example you bind the container internal data folder, /data to your local folder /var/warp10.

You must use the same --volume option in all your other docker commands on Warp 10™ image.

⁠For Windows users

First, you have to install Docker⁠ and optionally DockStation⁠.

docker run --volume=c:\\warp10:/data -p 8080:8080 -p 8081:8081 -d -i warp10io/warp10:tag