nikepan/clickhouse-bulk repository overview

This is official image of https://github.com/nikepan/clickhouse-bulk⁠

⁠Features

• Group n requests and send to any of ClickHouse server
• Sending collected data by interval
• Tested with VALUES, TabSeparated formats
• Supports many servers to send
• Supports query in query parameters and in body
• Supports other query parameters like username, password, database
• Supports basic authentication

⁠How to use

⁠docker

docker run -e "CLICKHOUSE_SERVERS=http://1.2.3.4:8123,http://3.4.5.6:8123" -p 8124 clickhouse-bulk

⁠docker-compose

clickhouse-bulk:
  image: clickhouse-bulk
  ports:
   - "8124"
  environment:
   - CLICKHOUSE_SERVERS=http://1.2.3.4:8123

⁠Available variaibles

• CLICKHOUSE_SERVERS - comma separated list of servers
• CLICKHOUSE_FLUSH_COUNT - count of rows for insert
• CLICKHOUSE_FLUSH_INTERVAL - insert interval
• DUMP_CHECK_INTERVAL - interval of resend dumps
• CLICKHOUSE_DOWN_TIMEOUT - wait time if server is down
• CLICKHOUSE_CONNECT_TIMEOUT - clickhouse server connect timeout

⁠Metrics

manual check main metrics curl -s http://127.0.0.1:8124/metrics | grep "^ch_"

• ch_bad_servers 0 - actual count of bad servers
• ch_dump_count 0 - dumps saved from launch
• ch_queued_dumps 0 - actual dump files id directory
• ch_good_servers 1 - actual good servers count
• ch_received_count 40 - received requests count from launch
• ch_sent_count 1 - sent request count from launch

⁠Tips

For better performance words FORMAT and VALUES must be uppercase.