Public | Automated Build

Last pushed: 9 days ago
Short Description
Latest Kafka - clusterable
Full Description

About:

Docker image based on openjdk:8-jre-alpine

Additional Software:

Usage:

The image provides a clusterable Kafka broker.

As a minimum the following environment variables must be set:

  1. KAFKA_BROKER_ID
  2. KAFKA_ADVERTISED_HOST_NAME
  3. KAFKA_ZOOKEEPER_CONNECT

So, assuming your Docker host is 172.17.8.101, has Zookeeper running and should now run Kafka as well, execute the following:

docker run -d -e KAFKA_BROKER_ID=1 -e KAFKA_ADVERTISED_HOST_NAME=172.17.8.101 -e KAFKA_ZOOKEEPER_CONNECT=172.17.8.101 digitalwonderland/kafka

(if you are looking for a clusterable Zookeeper Docker image, feel free to use digitalwonderland/zookeeper)

Additional Configuration

Configuration parameters can be provided via environment variables starting with KAFKA_. Any matching variable will be added to Kafkas server.properties by

  1. removing the KAFKA_ prefix
  2. transformation to lower case
  3. replacing any occurences of _ with .

For example an environment variable KAFKA_NUM_PARTITIONS=3 will result in num.partitions=3 within server.properties.

Evaluated Parameters

Any environment variable starting with KAFKA_ and ending with _COMMAND will be first evaluated and the result saved in an environment variable without the trailing _COMMAND.

For example an environment variable KAFKA_ADVERTISED_HOST_NAME_COMMAND=hostname will export KAFKA_ADVERTISED_HOST_NAME with the value obtained by running hostname command inside the container.

Docker Pull Command
Owner
digitalwonderland
Source Repository

Comments (4)
digitalwonderland
9 months ago

Please do not use the comment section to post issues since I almost never read those.

Instead please simply raise an issue at Github:

https://github.com/digital-wonderland/docker-kafka/issues

Thank you for understanding.

jiwanrai
a year ago

Appears the issue is related to the default log directory ('/var/lib/kafka' as per settings in /usr/local/sbin/start.sh). Even after clearing this directory, kafka fails to launch, with the "String index out of range" issue.

I've worked around the issue temporarily by setting the KAFKA_LOG_DIRS variable to /var/log/kafka.

i.e. my run command is something similar toL

docker run -d -p 9092:9092 -e KAFKA_BROKER_ID=1 -e KAFKA_LOG_DIRS=/var/log/kakfa -e KAFKA_ADVERTISED_HOST_NAME=172.17.8.101 -e KAFKA_ZOOKEEPER_CONNECT=172.17.8.101 digitalwonderland/kafka

timvw
2 years ago

I have the same exception ;(

Trying to remove files under /var/log/kafka does not help ;(

etspaceman
2 years ago

This build appears to be broken due to the most recent push. Starting a container now results in the following error:

[2015-09-16 18:21:54,128] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(Unknown Source)
at kafka.log.Log$.parseTopicPartitionName(Log.scala:833)
at kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$7$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:138)
at kafka.utils.Utils$$anon$1.run(Utils.scala:54)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)