Sign inSign up

moukoublen/kafka

By moukoublen

•Updated over 2 years ago

kafka docker image

Image
0

1.4K

moukoublen/kafka repository overview

⁠Kafka Docker

Kafka docker image using OpenJDK and more specifically eclipse-temurin:17-jre-jammy as base image. Currently images are being produced for these os/arch:

  • linux/amd64
  • linux/arm/v7
  • linux/arm64/v8

Docker Hub: https://hub.docker.com/r/moukoublen/kafka⁠.

GitHub: https://github.com/moukoublen/docker-kafka⁠.

Based on the excellent work of wurstmeister/kafka⁠

⁠Docker hub image tags

Schema: <scala version>-<kafka version>

⁠Tags
  • 3.4.1-2.13, latest
  • 3.4.1-2.12
  • 3.4.0-2.13
  • 3.4.0-2.12
  • 3.3.2-2.13
  • 3.3.2-2.12
  • 3.3.1-2.13
  • 3.3.1-2.12
  • 2.8.2-2.13
  • 2.8.2-2.12

⁠Configs source CONFIG_SOURCE

⁠Environment variables (env)

In order to use environment variables as configuration you must initialize env var CONFIG_SOURCE with env value (CONFIG_SOURCE: env).

Use properties from broker config⁠ directly as environment variables with server. prefix. Also log4j properties can be used direclty using log4j. prefix.

Table of environment variables prefix and destination file.

PrefixDestination file
server${KAFKA_HOME}/config/server.properties
log4j${KAFKA_HOME}/config/log4j.properties"
version: '3.8'
services:

  kafka:
    image: moukoublen/kafka
    ports:
      - "9094:9094"
    environment:
      KAFKA_HEAP_OPTS: -Xmx2g -Xms2g
      CONFIG_SOURCE: env
      server.broker.id: 1
      server.zookeeper.connect: zookeeper:2181
      server.listeners: INSIDE://kafka:9092,OUTSIDE://kafka:9094
      server.advertised.listeners: INSIDE://kafka:9092,OUTSIDE://localhost:9094
      server.listener.security.protocol.map: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      server.inter.broker.listener.name: INSIDE
      server.auto.create.topics.enable: "false"
      log4j.logger.kafka: DEBUG
      log4j.logger.org.apache.kafka: DEBUG
    restart: on-failure
⁠Files (files)

By setting CONFIG_SOURCE to files (e.g. CONFIG_SOURCE: files) conf files remain untouched and can be passed as link.

version: '3.8'
services:

  kafka:
    image: moukoublen/kafka
    ports:
      - "9094:9094"
    environment:
      KAFKA_HEAP_OPTS: -Xmx2g -Xms2g
      CONFIG_SOURCE: files
    volumes:
      - $PWD/config/server.properties:/opt/kafka/config/server.properties
      - $PWD/config/log4j.properties:/opt/kafka/config/log4j.properties
    restart: on-failure

⁠Topics seeding in docker compose

An easy way to create topics using docker compose is this:

services:
  zookeeper:
    # ...
  kafka:
    # ...
    environment:
      server.listeners: INSIDE://kafka:9092,OUTSIDE://kafka:9094
      server.advertised.listeners: INSIDE://kafka:9092,OUTSIDE://localhost:9094
      # ...

  topics-seed:
    image: moukoublen/kafka:latest
    restart: on-failure
    command: >
      bash -c "set -ex
      kafka-topics.sh --bootstrap-server 'INSIDE://kafka:9092' --create --topic samples1 --if-not-exists
      "

Check single-kafka.yml⁠ and kraft-cluster.yml⁠ for example.

⁠Docker compose examples

Single kafka node

docker compose -f compose/single-kafka.yml up

Cluster with zookeeper

docker compose -f compose/cluster-kafka.yml up

Kraft cluster (with external mounting of kafka log directory)

mkdir -p ./compose/logs/kraft-kafka{1,2,3}
docker compose -f compose/kraft-cluster.yml up

Tag summary

Content type

Image

Digest

sha256:83afb4a22…

Size

238.1 MB

Last updated

almost 3 years ago

docker pull moukoublen/kafka