alexsuch/kafka repository overview

⁠Apache Kafka and Zookeeper

This Docker container allows you to launch instances of Apache Kafka and Apache Zookeeper.

The can be launched in the same container on in a clusterized way.

⁠Environment variables

⁠MODE

It accepts three possible values:

• ZOOKEEPER_AND_KAFKA. This mode will launch an instance of zookeeper and another one of kafka in the container
• ZOOKEEPER. This mode will launch only an instance of zookeeper in the container
• KAFKA. This mode will launch only an instance of kafka in the container

⁠Zookeeper

• ZOOKEEPER_PORT

Maps to clientPort property. Default value is 2181.

• ZOOKEEPER_DATA_DIR

Maps to dataDir property. Default value is /tmp/zookeeper

• ZOOKEEPER_INIT_LIMIT

Maps to initLimit property. Default value is 5

• ZOOKEEPER_SYNC_LIMIT

Maps to syncLimit property. Default value is 2

• ZOOKEEPER_TICK_TIME

Maps to tickTime property. Default value is 2000

⁠Kafka

• KAFKA_BROKER_ID

Maps to broker.id property. Default value is 0. If you create a cluster of kafka brokers this number must not be repeated.

• KAFKA_PORT

This will add the entry listeners=PLAINTEXT://$HOSTNAME:$KAFKA_PORT to the kafka properties. Its default value is 9092.

• KAFKA_LOG_DIRS

Maps to log.dirs property. Default value is /tmp/kafka-logs

• KAFKA_NUM_PARTITIONS

Maps to num.partitions property. Default value is 1.

• KAFKA_AUTO_CREATE_TOPICS_ENABLE

Maps to auto.create.topics.enable property. Default value is true.

• KAFKA_DEFAULT_REPLICATION_FACTOR

Maps to default.replication.factor property. Default value is 1.

• KAFKA_LOG_RETENTION_HOURS

Maps to log.retention.hours property. Default value is 168.

• KAFKA_ZOOKEEPER_CONNECT

Maps to zookeeper.connect property. Default value is localhost:2181. It accepts a comma-separated list of zookeeper instances in a host:port form.

⁠Running

⁠Standalone

The default mode for the container is ZOOKEEPER_AND_KAFKA. So, for running an instance with all servers the command is:

docker run -p9092:9092 -p2181:2181 -it --rm --hostname kafka alexsuch/kafka:0.11.0.0

If running locally, and/or don't want to map the kafka entry in your /etc/hosts file, you can run:

docker run -p9092:9092 -p2181:2181 -it --rm --hostname localhost alexsuch/kafka:0.11.0.0

If using docker-compose, here you have⁠ an example that achieves the same infrastructure.

⁠Clustering

There are some docker-compose files in the docker-compose-samples⁠ folder.

• docker-compose-one-zk-one-kafka.yml⁠ creates an istance of kafka and zookeeper in two separated containers.
• docker-compose-one-zk-three-kafkas.yml⁠ creates three istances of kafka and one zookeeper in separated containers. The KAFKA_DEFAULT_REPLICATION_FACTOR is set to 1 so only one node will replicate.
• docker-compose-one-zk-two-kafkas.yml⁠ is the same scenario as before but with two kafka instances.
• docker-compose-two-zk-three-kafkas.yml⁠ creates three istances of kafka and two zookeeper nodes in separated containers. The KAFKA_DEFAULT_REPLICATION_FACTOR is set to 2, so all brokers will contain replicas of the topics. Besides, the KAFKA_ZOOKEEPER_CONNECT contains the two zookeeper instances addresses.