telix/wl-kafka-manager repository overview

⁠KafkaManager

This is a standalone docker image which contains frontend (Angular) and backend (Spring Boot app) parts.

Depending your needs and system you may want to provide additional services such as:

• Database: Required when you need to connect to multiple Kafka cluster
• Elasticsearch: Required if you want to keep mettrics for longer periods. Without it, metrics are kept for the user session duration (ie, the time you are logged into the application)

⁠Run the container

To start the Kafka manager application, you must run the docker image with several parameters which are described bellow.

For information, this docker image contains all application parts, ie, frontend (Angular website) and backend (Spring Boot application).

⁠Example to start docker container

In this example, we will launch the "KafkaManager" application in "Restricted mode"

docker run -d --name kafka-manager -p 8001:8001 -v /tmp/kafka-manager:/srv/log -e "JAVA_OPTS=-Dcluster.name=localCluster -Dcluster.kafkaVersion=2.4.2 -Dcluster.zkAddr=127.0.0.1:2181 -Dcluster.brokerAddrs[0].address=127.0.0.1 -Dcluster.brokerAddrs[0].kafkaPort=9092 -Dcluster.brokerAddrs[0].jmxPort=9093" telix/wl-kafka-manager:latest

In this example, we will launch the "KafkaManager" application in "MySQL mode"

docker run -d --name kafka-manager -p 8001:8001 -v /tmp/kafka-manager:/srv/log -e "JAVA_OPTS=-Ddatabase.enable=true -Dspring.datasource.url=jdbc:mysql://127.0.0.1:3306/kafka_manager?serverTimezone=Europe/Paris -Dspring.datasource.username=root -Dspring.datasource.password=root" telix/wl-kafka-manager:latest

⁠General options

• logging.level.com.worldline.kafka.kafkamanager=DEBUG

⁠Database options (default values)

• -Ddatabase.enable = true
• -Dspring.datasource.url = jdbc:mysql://127.0.0.1:3306/kafka_manager?serverTimezone=Europe/Paris
• -Dspring.datasource.username = root
• -Dspring.datasource.password = root
• elasticsearch.enable = true

⁠Restricted mode options (Without database)

• -Ddatabase.enable = false
• -Dcluster.name = "YOUR_CLUSTER_NAME"
• -Dcluster.kafkaVersion = "YOUR_KAFKA_VERSION" (Optional)
• -Dcluster.zkAddr = "_YOUR_ZOOKEEPER_ADDR_WITH_PORT"
• -Dcluster.kafkaConnectAddr = "YOUR_KAFKA_CONNECT_ADDR"
• -Dcluster.brokerAddrs[X].address = "YOUR_BROKER_ADDR"
• -Dcluster.brokerAddrs[X].kafkaPort = "YOUR_BROKER_PORT"
• -Dcluster.brokerAddrs[X].jmxPort = "YOUR_BROKER_JMX_PORT"

⁠Kafka Options (Default value)

• kafka.consumerservice.poll.duration="500" #in ms

Example with override (5Sec)

⁠Authentication

Three authenticiation modes are supported:

⁠JWT Configuration

• authentication.jwt.expiration= 14400
• authentication.jwt.secret= "YOUR_BASE64_SECRET"

⁠OpenId auth

• authentication.open-id.url= "URL_OPENID_CONNECT"
• authentication.open-id.clientId= "YOUR_REALME"
• authentication.open-id.secret= "YOUR_SECRET"
• authentication.open-id.adminRoleMapping= "ADMIN_ROLE_MAPPING"
• authentication.open-id.adminUsernameMapping= "ADMIN_IDS"
• authentication.open-id.adminWhiteList= "ADMIN_IDS"

⁠InMemory auth

• authentication.in-memory.enabled=true
• authentication.in-memory.userLogin= "user"
• authentication.in-memory.userPasswordHash= "USER_BCRYPT_HASH_PWD"
• authentication.in-memory.adminLogin= "admin"
• authentication.in-memory.adminPasswordHash= "ADMIN_BCRYPT_HASH_PWD"

⁠ElasticSearch Options (Default values)

• -Delasticsearch.enable= false
• elasticsearch.url= "SERVER:PORT"

⁠Metrics / Monitoring options (Default values)

⁠Metrics

• -Dmetrics.enable = true
• -Dmetrics.schedule = true
• -Dmetrics.cron-async-storage = 0 */1 * * * *
• -Dmetrics.clean.cron-clean-storage = 0 */1 * * * *
• -Dmetrics.clean.clean-time = 30

⁠Monitoring

• -Dmonitoring.enable= true

⁠Event Options

• -Devent.clean.cron-clean-storage = 0 */1 * * * *
• -Devent.clean.clean-time = 30

⁠Kafka Connect

• -Dkafka-connect.enable = true

⁠License

This project is distributed under Apache 2.0 license.

Kafka Manager uses the following external libraries :

• AdminLTE⁠ under MIT license
• AmCharts4⁠ covered by a freeware license

As with all Docker images, these likely also contain other software which may be under other licenses (such as Bash, etc from the base distribution, along with any direct or indirect dependencies of the primary software being contained).

As for any pre-built image usage, it is the image user's responsibility to ensure that any use of this image complies with any relevant licenses for all software contained within.

⁠Credits

Original project from Worldline FS Lab 2021

⁠Authors

• Romain Thely
• Baptiste Coulaud
• Flavien Flandrin
• Elliot Humbert