pnixx/clickhouse-queue repository overview

⁠Clickhouse Queue

Clickhouse insert buffer queue using RabbitMQ STOMP protocol.

Run container with shared tmp directory:

docker run -it -v /path/to/tmp:/var/www/clickhouse-queue/tmp --name clickhouse-queue pnixx/clickhouse-queue

Environment variables:

• CLICKHOUSE_HOST - ClickHouse host for inserts, default localhost
• CLICKHOUSE_PORT - ClickHouse port for inserts, default 8123
• CLICKHOUSE_DB - ClickHouse database for inserts, default default
• CLICKHOUSE_USER - ClickHouse user, optional
• CLICKHOUSE_PASSWORD - ClickHouse password, optional
• CLICKHOUSE_SSL - ClickHouse need use HTTPS connection instead HTTP, default false
• RABBIT_HOST - RabbitMQ host for subscribe, default localhost
• RABBIT_STOMP_PORT - RabbitMQ STOMP port, default 61613
• RABBIT_USER - RabbitMQ STOMP login user, default guest
• RABBIT_PASSWORD - RabbitMQ STOMP login password, default guest
• RABBIT_QUEUE - RabbitMQ queue for buffer data, default clickhouse
• RABBIT_VHOST - RabbitMQ vhost, default /
• MAX_DELAY - time flush data to ClickHouse, default 5
• MAX_FILE_SIZE - max buffer size before flush to ClickHouse, default 5242880
• LOG_LEVEL - output log level, default warning

⁠Message structure

Header table: to insert data into the specified table

Body: JSON as string row data

{"column1": "data1", "column2": "data2"}

Example publish:

rabbitmqadmin publish routing_key=clickhouse payload='{"column1":"data1","column2":"data2"}' properties='{"headers":{"table":"messages"}}'

⁠Donations

Donations to this project are going directly to PNixx⁠, the original author of this project:

• BTC address: 1H3rhpf7WEF5JmMZ3PVFMQc7Hm29THgUfN
• ETH address: 0x6F094365A70fe7836A633d2eE80A1FA9758234d5
• XMR address: 42gP71qLB5M43RuDnrQ3vSJFFxis9Kw9VMURhpx9NLQRRwNvaZRjm2TFojAMC8Fk1BQhZNKyWhoyJSn5Ak9kppgZPjE17Zh
• TON address: UQB5RntjBNMA6qXjuqTrl2TcqceNu3vMOj1AyyW3eeGGI-Qd