atendeindustries/tstorage repository overview

TStorage is a high-performance time-series database, optimized for measurement data — especially from sources such as IoT devices. It's built to handle versioned, timestamped records and supports fine-grained queries. A unique characteristic of TStorage is a 5-dimensional key that is well-suited to meet the needs of measurement systems. The key's components are listed below:

• cid: the ID of the cluster/user,
• mid: the ID of the measurement device,
• moid: the measured quantity,
• cap: the time the measurement was captured,
• acq: the time the measurement was acquired by the system. Due to the key-value separation, the key can be associated with any kind of value; it can be a scalar, a vector, a matrix, or a compressed or encrypted file.

TStorage provides consistency guarantees for GET and PUT requests. It might happen that GET requests wants to read data from the same key range in which some PUT request is writing data. In general, it is not obvious what user might expect as a result of these GET operations. TStorage provides following consistency guarantees to facilitate correct data processing in this case:

• Each successful GET request returns information about last acq - acq0, data version to which TStorage guarantees that all data are persisted in a given key range and no new data will be persisted in this range in the future. New data for that range will be stamped with higher version of acq time than acq0. Records returned by GET request are limited to key range with "trimmed" acqMax time to acq0. This also means that acq0 may be used to perform "incremental" reads, allowing users to perform actions on new data periodically.
• Each subsequent GET request with the same range trimmed to acq0 will return the same result.
• For two GET operations performed one after another, if the second GET request has a key range that is a subset of the first one's key range, then the second GET returns all records returned by the first GET in that range.

Please note that above statements apply only under normal system operation.

⁠Single node version

While TStorage is a distributed database, this Docker image contains single node version - free for both personal as well as commercial use. Currently we support only 64-bit architectures (x86-64 and ARM 64). If you find any bugs, have problems with running our instance or want to share your feedback feel free to contact us at: [email protected]⁠.

Additionally, you may also write to us if you are interested in a full, distributed version of TStorage with additional capabilities. Both variations are compatible with each other from the tooling perspective, so you may freely test our demo database for your use-case and make decision about switching to full version at a convenient time.

⁠Getting started

• tstorage.eu⁠ - official TStorage website
  • docs.tstorage.eu⁠ - TStorage documentation
• https://github.com/atendeindustries/tstorage-clients⁠ - client libraries
  • https://github.com/atendeindustries/tstorage-clients/tree/master/python⁠ - TStorage Python client example usage

⁠Simplest way to use single node TStorage

docker run --name tstorage -d -p 2025:2025 -v /etc/localtime:/etc/localtime tstorage:{tag}

⁠Data persistency and Unpriveleged image

Image is designed to run TStorage database as non-root user.
Therefore, when using data persistence, it is necessary to ensure appropriate permissions.
The best practice is to use a docker volume.
To make data persistant you need to mount volume to container /sacks pat

docker volume create tstorage_data
docker run -d -v tstorage_data:/sacks tstorage:{tag}

⁠Accessing Logs

Logs are written to /var/log/tstorage/ inside container. Binding volume while container creating can give possibility to easily access them:

docker volume create tstorage_logs
docker run -d -v tstorage_logs:/var/log/tstorage tstorage:{tag}

⁠TStorage parameters configuration

Using environment variables you can configure TStorage parameters

docker run -d -e tstorage_cid_log2_width=0 -v ./data:/sacks tstorage:{tag}

More about possible parameters can be found in documentation⁠.

⁠Easly cleanup procedure

docker stop tstorage
docker rm tstorage
docker volume rm tstorage_data tstorage_logs

⁠Example compose file

volumes:
  tstorage_data:
  tstorage_logs:

services:
  tstorage:
    container_name: tstorage_database
    image: tstorage:{tag}
    volumes: 
      - tstorage_data:/sacks
      - tstorage_logs:/var/log/tstorage
      - /etc/localtime:/etc/localtime
    ports:
      - 2025:2025
    environment:
        tstorage_cid_log2_width: 0
        tstorage_timeout_ms: 350000