lshegll/kafka-offset-exporter

By lshegll

•Updated about 7 years ago

Fork echojc/kafka-offset-exporter

Image

8.8K

Overview Tags

lshegll/kafka-offset-exporter repository overview

⁠kafka_exporter

Kafka exporter for Prometheus. For other metrics from Kafka, have a look at the JMX exporter⁠.

⁠Table of Contents

Compatibility⁠
Dependency⁠
Download⁠
Compile⁠
- Build Binary⁠
- Build Docker Image⁠
Run⁠
- Run Binary⁠
- Run Docker Image⁠
Flags⁠
- Notes⁠
Metrics⁠
Grafana Dashboard⁠
Contribute⁠
Donation⁠
License⁠

⁠Compatibility

Support Apache Kafka⁠ version 0.10.1.0 (and later).

⁠Dependency

⁠Download

Binary can be downloaded from Releases⁠ page.

⁠Compile

⁠Build Binary

make

⁠Build Docker Image

make docker

⁠Docker Hub Image

docker pull danielqsj/kafka-exporter:latest

It can be used directly instead of having to build the image yourself. (Docker Hub danielqsj/kafka-exporter⁠)

⁠Run

⁠Run Binary

kafka_exporter --kafka.server=kafka:9092 [--kafka.server=another-server ...]

⁠Run Docker Image

docker run -ti --rm -p 9308:9308 danielqsj/kafka-exporter --kafka.server=kafka:9092 [--kafka.server=another-server ...]

⁠Flags

This image is configurable using different flags

Flag name	Default	Description
kafka.server	kafka:9092	Addresses (host:port) of Kafka server
kafka.version	1.0.0	Kafka broker version
sasl.enabled	false	Connect using SASL/PLAIN
sasl.handshake	true	Only set this to false if using a non-Kafka SASL proxy
sasl.username		SASL user name
sasl.password		SASL user password
tls.enabled	false	Connect using TLS
tls.ca-file		The optional certificate authority file for TLS client authentication
tls.cert-file		The optional certificate file for client authentication
tls.key-file		The optional key file for client authentication
tls.insecure-skip-tls-verify	false	If true, the server's certificate will not be checked for validity
topic.filter	.*	Regex that determines which topics to collect
group.filter	.*	Regex that determines which consumer groups to collect
web.listen-address	:9308	Address to listen on for web interface and telemetry
web.telemetry-path	/metrics	Path under which to expose metrics
log.level	info	Only log messages with the given severity or above. Valid levels: [debug, info, warn, error, fatal]
log.enable-sarama	false	Turn on Sarama logging

⁠Notes

Boolean values are uniquely managed by Kingpin⁠. Each boolean flag will have a negative complement: --<name> and --no-<name>.

For example:

If you need to disable sasl.handshake, you could add flag --no-sasl.handshake

⁠Metrics

Documents about exposed Prometheus metrics.

For details on the underlying metrics please see Apache Kafka⁠.

⁠Brokers

Metrics details

Name	Exposed informations
`kafka_brokers`	Number of Brokers in the Kafka Cluster

Metrics output example

# HELP kafka_brokers Number of Brokers in the Kafka Cluster.
# TYPE kafka_brokers gauge
kafka_brokers 3

⁠Topics

Metrics details

Name	Exposed informations
`kafka_topic_partitions`	Number of partitions for this Topic
`kafka_topic_partition_current_offset`	Current Offset of a Broker at Topic/Partition
`kafka_topic_partition_oldest_offset`	Oldest Offset of a Broker at Topic/Partition
`kafka_topic_partition_in_sync_replica`	Number of In-Sync Replicas for this Topic/Partition
`kafka_topic_partition_leader`	Leader Broker ID of this Topic/Partition
`kafka_topic_partition_leader_is_preferred`	1 if Topic/Partition is using the Preferred Broker
`kafka_topic_partition_replicas`	Number of Replicas for this Topic/Partition
`kafka_topic_partition_under_replicated_partition`	1 if Topic/Partition is under Replicated

Metrics output example

# HELP kafka_topic_partitions Number of partitions for this Topic
# TYPE kafka_topic_partitions gauge
kafka_topic_partitions{topic="__consumer_offsets"} 50

# HELP kafka_topic_partition_current_offset Current Offset of a Broker at Topic/Partition
# TYPE kafka_topic_partition_current_offset gauge
kafka_topic_partition_current_offset{partition="0",topic="__consumer_offsets"} 0

# HELP kafka_topic_partition_oldest_offset Oldest Offset of a Broker at Topic/Partition
# TYPE kafka_topic_partition_oldest_offset gauge
kafka_topic_partition_oldest_offset{partition="0",topic="__consumer_offsets"} 0

# HELP kafka_topic_partition_in_sync_replica Number of In-Sync Replicas for this Topic/Partition
# TYPE kafka_topic_partition_in_sync_replica gauge
kafka_topic_partition_in_sync_replica{partition="0",topic="__consumer_offsets"} 3

# HELP kafka_topic_partition_leader Leader Broker ID of this Topic/Partition
# TYPE kafka_topic_partition_leader gauge
kafka_topic_partition_leader{partition="0",topic="__consumer_offsets"} 0

# HELP kafka_topic_partition_leader_is_preferred 1 if Topic/Partition is using the Preferred Broker
# TYPE kafka_topic_partition_leader_is_preferred gauge
kafka_topic_partition_leader_is_preferred{partition="0",topic="__consumer_offsets"} 1

# HELP kafka_topic_partition_replicas Number of Replicas for this Topic/Partition
# TYPE kafka_topic_partition_replicas gauge
kafka_topic_partition_replicas{partition="0",topic="__consumer_offsets"} 3

# HELP kafka_topic_partition_under_replicated_partition 1 if Topic/Partition is under Replicated
# TYPE kafka_topic_partition_under_replicated_partition gauge
kafka_topic_partition_under_replicated_partition{partition="0",topic="__consumer_offsets"} 0

⁠Consumer Groups

Metrics details

Name	Exposed informations
`kafka_consumergroup_current_offset`	Current Offset of a ConsumerGroup at Topic/Partition
`kafka_consumergroup_lag`	Current Approximate Lag of a ConsumerGroup at Topic/Partition

Metrics output example

# HELP kafka_consumergroup_current_offset Current Offset of a ConsumerGroup at Topic/Partition
# TYPE kafka_consumergroup_current_offset gauge
kafka_consumergroup_current_offset{consumergroup="KMOffsetCache-kafka-manager-3806276532-ml44w",partition="0",topic="__consumer_offsets"} -1

# HELP kafka_consumergroup_lag Current Approximate Lag of a ConsumerGroup at Topic/Partition
# TYPE kafka_consumergroup_lag gauge
kafka_consumergroup_lag{consumergroup="KMOffsetCache-kafka-manager-3806276532-ml44w",partition="0",topic="__consumer_offsets"} 1

⁠Grafana Dashboard

Grafana Dashboard ID: 7589, name: Kafka Exporter Overview.

For details of the dashboard please see Kafka Exporter Overview⁠.

⁠Contribute

If you like Kafka Exporter, please give me a star. This will help more people know Kafka Exporter.

Please feel free to send me pull requests⁠.

⁠Donation

Your donation will encourage me to continue to improve Kafka Exporter. Support Alipay donation.

⁠License

Code is licensed under the Apache License 2.0⁠.

Tag summary

Recent tags

Content type

Image

Digest

Size

5.9 MB

Last updated

about 7 years ago

docker pull lshegll/kafka-offset-exporter