Public Repository

Last pushed: 2 years ago
Short Description
Kafka-ELK streaming PoC.
Full Description

Thanks to qnib (https://hub.docker.com/r/qnib/elk/) for initial docker


docker-compose

elk-kafka:
  image: intropro/kelk:0.7
  ports:
   - "9200:9200"
   - "5514:5514"
   - "55514:55514/udp"
   - "5601:5601"
   - "8080:80"
   - "8500:8500"
   - "9092:9092"
  environment:
  - DC_NAME=dc1
  - RUN_SERVER=true
  - BOOTSTRAP_CONSUL=true
  - COLLECT_METRICS=false
  - FORWARD_TO_LOGSTASH=false
#  - KAFKA_ADVERTISED_HOST_NAME=172.17.42.1
#  - KAFKA_ADVERTISED_PORT=9092
  dns: 127.0.0.1
  hostname: elk
  volumes: 
    - /tmp/ingest:/ingest:rw
  privileged: true

Check status of services on http://localhost:8500/ (Consul)
Check Kibana index logstash-* on http://localhost:5601/
Drop AVRO files to /tmp/ingest on your host (not inside Docker !). Docker will start processing and streaming Avro messages into ELK
Logstash looks for timestamp in eventData + eventTime fields. If they are in the past, don't forget to change the Kibana range discovery window !

Docker Pull Command
Owner
intropro

Comments (0)