An HDP 2.5.3 based Docker image for Kafka Mirror Maker. This allows configuration and deployment of Kafka Mirror-Maker in Dockerized environments like Kubernetes/Openshift or AWS ECS.
This image is available from Docker hub however, if you would like to build it yourself here are the steps:
git clone https://github.com/srotya/docker-kafka-mirror-maker.git
cd docker-kafka-mirror-maker
docker build -t mirror-maker:latest .
Note: Docker is expected to be installed where you run the build
| Variable Name | Description | Defaults |
|---|---|---|
| DESTINATION | bootstrap.servers for the Destination Kafka Cluster | localhost:6667 |
| SOURCE | bootstrap.servers for the Source Kafka Cluster | localhost:6667 |
| WHITELIST | Topics to mirror | * |
| SECURITY | If kerberos is enabled, valid options: PLAINTEXT or SASL_PLAINTEXT | PLAINTEXT |
| GROUPID | Consumer group id for Kafka consumer | _mirror_maker |
| PRINCIPAL | Kerberos security principal name | kafka/[email protected] |
| KEYTAB_FILENAME | File name for Kerberos Keytab | mirror.keytab |
Image requires that the keytab file and krb5.conf be mounted if Kerberos is used when deploying this container.
Keytab files are expected in /etc/security/keytabs location
Krb5.conf file is expected in /etc/krb5.conf location
docker run -it -e DESTINATION=xxx.xxx.com:9092 -e SOURCE=xxx.xxx.com:9092 -e WHITELIST=<TOPIC NAME> mirror-maker:latest
docker run -it -e DESTINATION=xxx.xxx.com:6667 -e SECURITY=SASL_PLAINTEXT -e SOURCE=xxx.xxx.com:6667 -e WHITELIST=<TOPIC NAME> -e KEYTAB_FILENAME=kafka.service.keytab -e PRINCIPAL=kafka/[email protected] -v /etc/security/keytabs/:/etc/security/keytabs/ -v /etc/krb5.conf:/etc/krb5.conf mirror-maker:latest
Note: the image automatically applies Kerberos runtime configuration for Mirror Maker if SASL_PLAINTEXT security is configured
To deploy this image in Kubernetes here's the template.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: mirror-maker
spec:
replicas: 3
template:
metadata:
labels:
name: mirror-maker
spec:
containers:
- name: mirror-maker
image: ambuds/mirror-maker
imagePullPolicy: Always
env:
- name: "WHITELIST"
value: "*"
- name: "DESTINATION"
value: "localhost:9092"
- name: "SOURCE"
value: "localhost:9092"
Apache 2.0
Content type
Image
Digest
Size
277.3 MB
Last updated
over 9 years ago
docker pull ambuds/mirror-maker