Public | Automated Build

Last pushed: 2 years ago
Short Description
Dockerfile for Apache Kafka
Full Description

kafka-broker

Dockerfile for Apache Kafka

The image is available directly from https://registry.hub.docker.com/

##Pre-Requisites

  • install docker-compose https://docs.docker.com/compose/install/
  • modify the KAFKA_ADVERTISED_HOST_NAME in docker-compose.yml to match your docker host IP (Note: Do not use localhost or 127.0.0.1 as the host ip if you want to run multiple brokers.)
  • if you want to customise any Kafka parameters, simply add them as environment variables in docker-compose.yml, e.g. in order to increase the message.max.bytes parameter set the environment to KAFKA_MESSAGE_MAX_BYTES: 2000000. To turn off automatic topic creation set KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false'

##Usage

Start a cluster:

  • docker-compose up -d

Add more brokers:

  • docker-compose scale kafka=3

Stop a cluster:

  • docker-compose stop

Destroy a cluster:

  • docker-compose down

Monitor containers running in a cluster:

  • docker-compose ps

Stop container running in a cluster:

  • docker stop $container_name

Start container in a cluster:

  • docker start $container_name

##Generating kafa producer command

If you want to generate a kafka producer command in order to produce data from console for testing purpose, execute the next script:

  • sh create-producer.sh $topicName

##Generating kafka consumer command

If you want to generate a kafka consumer command to consume data from console for testing purpose, execute the next script:

  • sh create-consumer.sh $topicName

##Automatically create topics

If you want to have kafka-docker automatically create topics in Kafka during
creation, a KAFKA_CREATE_TOPICS environment variable can be
added in docker-compose.yml.

Here is an example snippet from docker-compose.yml:

    environment:
      KAFKA_CREATE_TOPICS: "Topic1:1:3,Topic2:1:1"

Topic 1 will have 1 partition and 3 replicas, Topic 2 will have 1 partition and 1 replica.

##Advertised hostname

You can configure the advertised hostname in different ways

  1. explicitly, using KAFKA_ADVERTISED_HOST_NAME
  2. via a command, using HOSTNAME_COMMAND, e.g. HOSTNAME_COMMAND: "route -n | awk '/UG[ \t]/{print $$2}'"

When using commands, make sure you review the "Variable Substitution" section in https://docs.docker.com/compose/compose-file/

If KAFKA_ADVERTISED_HOST_NAME is specified, it takes presendence over HOSTNAME_COMMAND

Docker Pull Command
Owner
cirobarradov
Source Repository

Comments (0)