Public | Automated Build

Last pushed: 2 years ago
Short Description
Simple time-series database
Full Description

tsdb

TSDB is a simple approach to implementing a time-series database. It
is partly inspired by InfluxDB, but writes its content in plain
text files. TSDB provides support for the Graphite API and the
InfluxDB API to write data into the database.

This is a work in progress. For example, there are currently no
client API implementation for retrieving data (even though there is an
implementation in the database code).

Design

TSDB divides its space into series (as in time series (of metrics)).
Each series will accept any number of samples as time passes. Time is
preferably implicit, i.e. at the time of writing, but can also be
specified. On disk, each series will be associated to a directory,
itself having as many directories as the series has ever had samples.
In each sample directory, there will be a number of timeblocks, where
the name can be directly mapped to the timestamp of the first sample
in that block. TSDB tries to limit the size of these blocks to ease
copying.

As time goes by and values for samples are written, these values will
be appended to a timeblocks and new timeblocks will be created
whenever necessary. If values are inserted in the middle of an
existing timeblock, the value will first be appended to the end of the
file representing the timeblock and the block will be marked as
"dirty" (TSDB uses the filename for flag representation). Later on,
dirty blocks will be reordered on disk.

Dependencies

In order to support the InfluxDB API, the example client uses the mini
HTTP server from [til][4]. It assumes that a copy of the til is
available in the lib directory (see the Dockerfile for an example).

[4]: http://code.google.com/p/efr-tools/source/browse/#svn%2Ftrunk%2Ftil

Internal API

To create a new database called mydb in the root directory
/opt/data (where you might want to place other databases), you would
call:

set db [::tdb::create -name mydb -root /opt/data]

This will return a command that will be used for all further access to
the database.

To start storing data for the series called example and create two
samples and their values, you would call:

$db store example mykey 10 anotherkey "a string of your choice"

The special keys time or timestamp are recognised as the time at
which the keys should be set, these are expressed in the number of
milliseconds since the epoch.

To retrieve all the values for a sample that have been put in the
database for the past 10 seconds, you would call:

$db samples example mykey -10000

To retrieve all the values for a sample between two specific dates,
you would call (note that the starting point is specified after the
end of the series, because of the relative case above. This might
change in the future).

$db samples example mykey 1424732064000 1424732054000

Client API

Graphite

The example client is able to listen for incoming writes expressed
using the textual carbon API. Graphite uses dots . to create a
hierarchy of metrics. The server will understand the top of the
hierarchy as the name of the series and the rest as the name of a
sample.

InfluxDB

The example client is able to receive HTTP post on the standard
InfluxDB port. There is no support for authentication at this point
(even though this is supported by the internal HTTP server).

Docker Component

The example server can be packaged as a docker component. To create it
run the following:

docker build -t efrecon/tsdb .

To run it with maximal verbosity (not recommended), execute the
following command. This will create a server that listen for incoming
client connections on the graphite default port (2003') and on the InfluxDB default port (8086). Both ports are exposed, as examplified below. The components is configured to use a database calleddb, located in the exported volume/data`.

docker run -d --name="tsdb" -p 2003:2003 -p 8086:8086 efrecon/tsdb -v 6
Docker Pull Command
Owner
efrecon
Source Repository