Sign inSign up

ashokhin/openshift-kafka

By ashokhin

•Updated over 1 year ago
Archived

Docker container based on Alpine, Java OpenJDK and official Apache Kafka + Apache Zookeeper.

Image
1

7.3K

ashokhin/openshift-kafka repository overview

⁠Apache Kafka for OpenShift

Run Apache Kafka⁠ and Apache ZooKeeper⁠ on OpenShift v3⁠.

This was developed for demonstration purposes. Use in environments where you do not need persistence in the face of failures or configuration tuning for scale.

The architecture is as follows:

  • 1 pod containing a container for Apache Kafka and Apache ZooKeeper
  • 1 service to access the containers

⁠Quick start

⁠Load resources (Templates, DeploymentConfigs, Services)
oc create -f https://raw.githubusercontent.com/ashokhin/openshift-kafka/master/resources.yaml
⁠Deploy the Apache Kafka + Apache Zookeeper pod
oc new-app apache-kafka
⁠Optional: Follow container logs

choose one of: [apache-kafka apache-zookeeper]

$ oc get pods
NAME                   READY     STATUS    RESTARTS   AGE
apache-kafka-1-ln40b   2/2       Running   0          1s

$ oc logs --follow apache-kafka-1-ln40b -c apache-kafka
[2017-08-10 12:25:17,652] INFO KafkaConfig values: 
	advertised.host.name = apache-kafka

$ oc logs --follow apache-kafka-1-ln40b -c apache-zookeeper
[2017-08-10 12:25:19,697] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)

...for specific diagnostic purpose log grepping could be useful :

$ oc logs --follow apache-kafka-1-ln40b -c apache-zookeeper | grep "socket connection"
[2017-08-10 12:28:25,291] INFO Accepted socket connection from /172.17.0.7:57822 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2017-08-10 12:28:25,324] INFO Closed socket connection for client /172.17.0.7:57822 which had sessionid 0x15dcc1b2ea40001 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-08-10 12:28:38,960] INFO Accepted socket connection from /172.17.0.7:57838 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2017-08-10 12:28:38,996] INFO Closed socket connection for client /172.17.0.7:57838 which had sessionid 0x15dcc1b2ea40002 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-08-10 12:30:33,112] INFO Accepted socket connection from /172.17.0.7:57996 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2017-08-10 12:30:33,332] INFO Closed socket connection for client /172.17.0.7:57996 which had sessionid 0x15dcc1b2ea40003 (org.apache.zookeeper.server.NIOServerCnxn)
⁠Follow the Apache Kafka Documentation Quick Start⁠
  1. Deploy a debugging container and connect to it
oc run -it --rm kafka-debug --image=ashokhin/openshift-kafka --command -- bash
  1. Create a topic
bin/kafka-topics.sh --create --zookeeper apache-kafka --replication-factor 1 --partitions 1 --topic test
  1. List topics
bin/kafka-topics.sh --list --zookeeper apache-kafka
  1. Send some messages
bin/kafka-console-producer.sh --broker-list apache-kafka:9092 --topic test <<EOF
foo
bar
baz
EOF
  1. Receive some messages
bin/kafka-console-consumer.sh --bootstrap-server apache-kafka:9092 --topic test --from-beginning

⁠Docker Hub⁠

⁠Credits

Tag summary

Content type

Image

Digest

sha256:18dca8316…

Size

177.7 MB

Last updated

over 1 year ago

docker pull ashokhin/openshift-kafka