thingful/iotstore-amd64 repository overview

This image contains an implementation of the encrypted datastore interface being developed as part of the DECODE project.

DECODE is a European Commission funded project exploring and piloting new technologies that give people more control over how they store, manage and use personal data generated online. More details on this project can be found at https://decodeproject.eu/⁠.

This component implements the datastore interface as defined in this repo: https://github.com/thingful/decode-protorepo⁠.

⁠Quick Reference

• Where to get help?

• Where to file issues: https://github.com/thingful/iotstore/issues⁠

• Maintained by: https://github.com/thingful⁠

• License: https://github.com/thingful/iotstore/blob/master/LICENSE⁠

⁠Running the image

To use the image it needs to have access to a PostgreSQL server in order to persist incoming data. This may be an existing server on your machine, but the simplest way to run the image is via docker compose. An example compose file is shown below:

version: '3'
services:
  postgres:
    image: postgres:10-alpine
    ports:
      - "5432:5432"
    restart: always
    volumes:
      - postgres_vol:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=password
      - POSTGRES_USER=decode
      - POSTGRES_DB=postgres

  datastore:
    image: thingful/iotstore-amd64:v0.1.1
    ports:
      - "8080:8080"
    restart: always
    environment:
      - IOTSTORE_DATABASE_URL=postgres://decode:password@postgres:5432/postgres?sslmode=disable
    depends_on:
      - postgres
    command: [ "server", "--verbose" ]

volumes:
  postgres_vol:

The above compose file starts postgresql and the datastore containers running in verbose mode using the default postgres system database. We will need to a little more work to run with other services, but this will let you test out the basic operations of the datastore.

To start the above configuration you need to create the file shown above in a directory, and name it docker-compose.yml, then execute the following command:

$ docker-compose up

After a few seconds the datastore should boot up, connect to postgres, execute any pending schema modification migrations against the database and then it will be listening on https://localhost:8080⁠.

⁠Testing the server

To verify that the server is running correctly, you can access the following URL:

http://localhost:8080/pulse⁠

which will return a simple plain text response of ok if it is running and able to connect to the database.

The server provides an API implemented via a tool called Twirp; this is a protocol buffer based API for which we will generate and share API clients for supported languages, however it is possible to test out the server from the command line via curl.

⁠Writing data via curl

An example of how we might write to the datastore using curl would be:

curl --request "POST" \
       --location "http://localhost:8080/twirp/decode.iot.datastore.Datastore/WriteData" \
       --header "Content-Type: application/json" \
       --silent \
       --data "{\"public_key\":\"abc123\",\"data\":\"${data}\"}"

where the $data variable contains some encrypted bytes we wish to store in the datastore.

⁠Reading data via curl

start_time=$(date -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ)

curl --request "POST" \
       --location "http://localhost:8080/twirp/decode.iot.datastore.Datastore/ReadData" \
       --header "Content-Type: application/json" \
       --silent \
       --data "{\"public_key\":\"abc123\", \"page_size\":3, \"start_time\": \"${start_time}\"}"