Public Repository

Last pushed: 11 days ago
Short Description
Linked Burrow
Full Description

https://github.com/linkedin/Burrow

the image generates the burrow cfg on the fly based on the following environment variables:

required vars

ZK_HOSTS: a space separated list of zookeeper hostsname:port

ZK_HOSTS=${ZK_HOSTS:?"** ZOOKEEPER_HOSTS must be set!"}

KAFKA_HOSTS: a space separated list of kafka hostsname:port

KAFKA_ZK_HOSTS: a space separated list of zookeeper hostsname:port that kafka uses.

KAFKA_ZK_PATH: The full path to the znode that is the root for the Kafka cluster. If Kafka is configured with a chroot path for Zookeeper, "/" will specify the root znode.

KAFKA_HOSTS=${KAFKA_HOSTS:?" KAFKA_HOSTS must be set!"}
KAFKA_ZK_HOSTS=${KAFKA_ZK_HOSTS:?"
KAFKA_ZK_HOSTS must be set!"}
KAFKA_ZK_PATH=${KAFKA_ZK_PATH:?"** KAFKA_ZK_PATH must be set!"}

optional vars

GENERAL_GROUP_BLACKLIST: If specified, consumer group names that match this REGEX are ignored, eg ^(console-consumer-|python-kafka-consumer-).*$

GENERAL_GROUP_BLACKLIST=${GENERAL_GROUP_BLACKLIST:-}

ZK_TIMEOUT: The connection timeout for Zookeeper sessions, in seconds. This value is also used for the Kafka cluster Zookeeper connections.

ZK_LOCK_PATH: A full path to the znode to use for locking between Burrow instances. The path will be created if it doesn't already exist.

ZK_TIMEOUT=${ZK_TIMEOUT:-6}
ZK_LOCK_PATH=${ZK_LOCK_PATH:-/burrow/notifier}

KAFKA_ZK_OFFSETS: Selects whether or not Zookeeper committed offsets for this Kafka cluster are collected

KAFKA_OFFSETS_TOPIC: The name of the consumer offsets topic. This should not need to be specified at all.

KAFKA_ZK_OFFSETS=${KAFKA_ZK_OFFSETS:-true}
KAFKA_OFFSETS_TOPIC=${KAFKA_OFFSETS_TOPIC:-__consumer_offsets}

TICKERS_BROKER_OFFSETS: [int] How often to refresh the broker HEAD (end of log) offsets for all partitions, in seconds.

TICKERS_BROKER_OFFSETS=${TICKERS_BROKER_OFFSETS:-60}

LAGCHECK_INTERVALS: [int] How many consumer offsets to store for a partition. This determines the window over which to evaluate consumer lag.

LAGCHECK_EXPIRE_GROUP: [int] How long to wait, in seconds, after a consumer has stopped committing offsets to remove it from Burrow's monitoring.

LAGCHECK_MIN_DISTANCE: [int] The minimum interval, in seconds, allowed between consumer offset commits. Commits under this interval are ignored.

LAGCHECK_ZK_INTERVAL: [int] How frequently, in seconds, to scan Zookeeper committed offsets for clusters that are configured for ZK offsets.

LAGCHECK_ZK_GROUP_REFRESH: [int] How frequently, in seconds, to refresh the list of consumer groups for which Zookeeper offsets are collected.

LAGCHECK_INTERVALS=${LAGCHECK_INTERVALS:-10}
LAGCHECK_EXPIRE_GROUP=${LAGCHECK_EXPIRE_GROUP:-604800}
LAGCHECK_MIN_DISTANCE=${LAGCHECK_MIN_DISTANCE:-1}
LAGCHECK_ZK_INTERVAL=${LAGCHECK_ZK_INTERVAL:-60}
LAGCHECK_ZK_GROUP_REFRESH=${LAGCHECK_ZK_GROUP_REFRESH:-300}

HTTP_SERVER_MODE: on|off

HTTP_SERVER_PORT: any valid port number

HTTP_SERVER_MODE=${HTTP_SERVER_MODE:-on}
HTTP_SERVER_PORT=${HTTP_SERVER_PORT:-8000}

Docker Pull Command
Owner
hooli

Comments (2)
edkellena
6 months ago

Also interested to see the Dockerfile/source for this somewhere, would like to add in notifier options!

cyclops23
7 months ago

Is the Dockerfile for this image available somewhere?