dajudge/kafkaproxy repository overview

⁠kafkaproxy

kafkaproxy is a reverse proxy for the wire protocol of Apache Kafka.

Since the Kafka wire protocol publishes the set of available broker endpoints from the brokers to the clients during client bootstrapping, the brokers must be configured to publish endpoints that are reachable from the client side. This can be a cumbersome restriction in several different situations, such as:

• Network topologies preventing direct access to the broker nodes
• Multiple networks from which broker nodes should be reachable
• DNS resolution restrictions when accessing TLS secured broker nodes
• Using the sidecar pattern for TLS termination in Kubernetes

This is where kafkaproxy comes into play and allows for transparent relaying of the Kafka wire protocol by rewriting the relevant parts of the communication where the brokers publish the endpoint names - with user-configurable endpoints where the the proxy instances can be reached.

⁠Run kafkaproxy in Docker

The following configuration parameters are mandatory:

• KAFKAPROXY_HOSTNAME: the hostname at which the proxy can be reached from the clients.
• KAFKAPROXY_BASE_PORT: the first port to be used by kafkaproxy.
• KAFKAPROXY_BOOTSTRAP_SERVERS: the comma separated list of initially mapped endpoints. This is usually the list of bootstrap brokers or a load balancer in front of the kafka brokers.

For example:

docker run \
    --net host \
    -e KAFKAPROXY_HOSTNAME=localhost \
    -e KAFKAPROXY_BASE_PORT=4000 \
    -e KAFKAPROXY_BOOTSTRAP_SERVERS=kafka:9092 \
    -d dajudge/kafkaproxy:0.0.10

Note: You will have to make the proxy ports (starting with KAFKAPROXY_BASE_PORT and incrementing from there) available from outside the container with -p PORT:PORT if you're not using --net host.

⁠github

Read more at the github page⁠