kaktuss/corrie repository overview

⁠Corrie

Reliable (with RabbitMQ) Clickhouse writer.

Notice: if you need performance, it is better idea to use Ruthie⁠ - based on Redis Cluster it has much more better performance, then Corrie, based on RabbitMQ.

⁠Configuration

⁠CORRIE_RABBITMQ_ADDR

RabbitMQ address in host:port format

CORRIE_RABBITMQ_ADDR=rabbitmq.example.com:5672

⁠CORRIE_RABBITMQ_VHOST

RabbitMQ virtual host to store queue

CORRIE_RABBITMQ_VHOST=corrie

⁠CORRIE_RABBITMQ_USER, CORRIE_RABBITMQ_PASSWORD

RabbitMQ user and password

CORRIE_RABBITMQ_USER=corrie
CORRIE_RABBITMQ_PASSWORD=somepassword

⁠CORRIE_CLICKHOUSE_ADDR

Primary ClickHouse address in host:port form

CORRIE_CLICKHOUSE_ADDR=clickhouse1.example.com:9000

⁠CORRIE_CLICKHOUSE_ALTADDRS

Comma separated list of alternative ClickHouse addresses to loadbalancing. Can be empty

CORRIE_CLICKHOUSE_ALTADDRS=clickhouse2.example.com:9000

⁠CORRIE_BATCH

Set batch size of ClickHouse writes.

CORRIE_BATCH=10000

⁠Run

docker run --rm -it kaktuss/corrie

⁠Write data

To write data use message⁠ package.

You can write data with nanachi RabbitMQ client (see example) or with any other client.

Pay attention, that Corrie uses sharded queue (with nanachi) hardcoded to use 3 shards. Shards count will be configurable later.