anavpdel/kafka-rest-proxy

By anavpdel

Updated about 1 month ago

KAFKA rest proxy for producing json message using SSL certificate

Image
Integration & Delivery
Message Queues
Developer Tools

78

Kafka REST Proxy for Producing JSON Messages Using SSL Certificate

Changes Summary:
  1. Non-SSL: When SSL is not needed, you only need to set KAFKA_SECURITY_PROTOCOL_ENABLE=false, and the other SSL-related environment variables can be omitted.

    Example for non-SSL:

    docker run -e KAFKA_SECURITY_PROTOCOL_ENABLE=false -it -p 9501:9501 anavpdel/kafka-rest-proxy
    
  2. Custom Response: If you need a custom response, you can set CUSTOM_RESPONSE_JSON in the environment.

    Example for custom response:

    -e CUSTOM_RESPONSE_JSON='{"code":0, "message":"Notified successfully."}'
    

    This will return:

    {
        "code": 0,
        "message": "Notified successfully."
    }
    
Using Docker CLI

To run the Kafka REST Proxy with SSL enabled:

docker run -it -p 9501:9501 \
  --name kafka-rest-proxy \
  -e KAFKA_SECURITY_PROTOCOL_ENABLE=true \
  -e KAFKA_SSL_KEY_LOCATION=/ssl/server.key \
  -e KAFKA_SSL_CERTIFICATE_LOCATION=/ssl/server.crt \
  -e KAFKA_SSL_CA_LOCATION=/ssl/ca.crt \
  -e KAFKA_BROKERS=kafka:9093 \
  anavpdel/kafka-rest-proxy

To run without SSL (disable SSL):

docker run -e KAFKA_SECURITY_PROTOCOL_ENABLE=false -it -p 9501:9501 anavpdel/kafka-rest-proxy

Note: The following environment variables are required when SSL is enabled:

  • KAFKA_SECURITY_PROTOCOL_ENABLE
  • KAFKA_SSL_KEY_LOCATION
  • KAFKA_SSL_CERTIFICATE_LOCATION
  • KAFKA_SSL_CA_LOCATION
Using Docker Compose
kafka-rest-proxy:
  container_name: hyperf-kafka-producer
  image: anavpdel/kafka-rest-proxy
  volumes:
    - ./ssl:/ssl
  ports:
    - 9508:9501
  stdin_open: true
  tty: true
  environment:
    - APP_ENV=dev
    - SCAN_CACHEABLE=false
    - KAFKA_SECURITY_PROTOCOL_ENABLE=false
    - KAFKA_SSL_KEY_LOCATION=anything
    - KAFKA_SSL_CERTIFICATE_LOCATION=anything
    - KAFKA_SSL_CA_LOCATION=anything
  networks:
    - internal

You can send a POST JSON payload to:

http://localhost:9501/topics/{topicName}

For example, POST to http://localhost:9501/topics/hyperf-rest-proxy:

{
    "something": "hello world"
}

This sends JSON data to hyperf-rest-proxy.

Additional Configuration
  • Default Memory Limit: 1GB
  • Default Timezone: Asia/Kathmandu

To customize these values, you can add:

-e memory_limit=4G -e timezone=asia/shanghai

Docker Pull Command

docker pull anavpdel/kafka-rest-proxy