Public | Automated Build

Last pushed: 7 months ago
Short Description
A simple docker image with both Kafka and Zookeeper.
Full Description

Kafka in Docker

This repository provides everything you need to run Kafka in Docker.

For convenience also contains a packaged proxy that can be used to get data from
a legacy Kafka 7 cluster into a dockerized Kafka 8.


The main hurdle of running Kafka in Docker is that it depends on Zookeeper.
Compared to other Kafka docker images, this one runs both Zookeeper and Kafka
in the same container. This means:

  • No dependency on an external Zookeeper host, or linking to another container
  • Zookeeper and Kafka are configured to work together out of the box


docker run -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=`docker-machine ip \`docker-machine active\`` --env ADVERTISED_PORT=9092 spotify/kafka
export KAFKA=`docker-machine ip \`docker-machine active\``:9092 --broker-list $KAFKA --topic test
export ZOOKEEPER=`docker-machine ip \`docker-machine active\``:2181 --zookeeper $ZOOKEEPER --topic test

Running the proxy

Take the same parameters as the spotify/kafka image with some new ones:

  • CONSUMER_THREADS - the number of threads to consume the source kafka 7 with
  • TOPICS - whitelist of topics to mirror
  • ZK_CONNECT - the zookeeper connect string of the source kafka 7
  • GROUP_ID - the to use when consuming from kafka 7
docker run -p 2181:2181 -p 9092:9092 \
    --env ADVERTISED_HOST=`boot2docker ip` \
    --env ADVERTISED_PORT=9092 \
    --env CONSUMER_THREADS=1 \
    --env TOPICS=my-topic,some-other-topic \
    --env ZK_CONNECT=kafka7zookeeper:2181/root/path \
    --env GROUP_ID=mymirror \

In the box

  • spotify/kafka

    The docker image with both Kafka and Zookeeper. Built from the kafka

  • spotify/kafkaproxy

    The docker image with Kafka, Zookeeper and a Kafka 7 proxy that can be
    configured with a set of topics to mirror.

Public Builds

Build from Source

docker build -t spotify/kafka kafka/
docker build -t spotify/kafkaproxy kafkaproxy/


  • Not particularily optimzed for startup time.
  • Better docs
Docker Pull Command
Source Repository

Comments (14)
5 months ago

Your image requires login to pull.

$ docker run -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST= --env ADVERTISED_PORT=9092 spotify/kafka
Unable to find image 'spotify/kafka:latest' locally
docker: Error response from daemon: Get unauthorized: incorrect username or password.
5 months ago

how to create topics while starting kafka from docker-compose

7 months ago

What is 'boot2docker ip'? Should it be the ip of the docker engine host machine?

9 months ago

Would there be any benefit to upgrading the version of kafka?

10 months ago

very good!

a year ago

After looking around I did:
cd /opt/kafka_2.11-

and found the commands related to kafka

Is that the recommended way ?

a year ago

newbie question:
if I login to a shell in the running kafka container, I don't see the kafka console producer or kafka console consumer commands available. How do I have to run the kafka producer consumer commands ?
I attached to the running container by
docker exec -it <hashid> bash

a year ago

regarding @ramwin comment:
You are right , it happened to me also
Just downloaded the repo and change kafka version to 0.9.0.o on docker file and all worked fine .

a year ago

Please add some information, what your changes include when you build a new version. The latest version seems to update the glibc vulnerability.

dpkg -s libc6
Version: 2.19-18+deb8u3

a year ago

<center><big>IMPORTANT WARNING</big></center>
This images use KAFKA version,
If you use kafka to create a producer, there is no problem.
But when you use kafka to create a consumer, it will raise error like this:
[2016-02-19 15:43:21,064] WARN [ConsumerFetcherThread-console-consumer-30554_wangxiang-PC-1455867762688-99645940-0-0], Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@2a1212a0. Possible cause: java.nio.BufferUnderflowException (kafka.consumer.ConsumerFetcherThread)
^CProcessed a total of 1 messages