Dockerfile creating ELK services (Elasticsearch/Logstash/Kibana)
Please use at least 2GB of RAM for this one... See #16
It's available on hub.docker.com, just pull it:
docker pull qnib/elk
- connects with consul, if available
- start sshd
- start logstash
- start diamond
- start StatsD
- start elasticsearch
- start nginx (kibana3)
- start kibana4
How to use kibana3 and kibana4 could be explored within this 'hello world' blog post.
To get the most out of it a carbon container might be added, but this will impose the question whether to go even further and distribute all the services.
Time mismatch in rsyslog
If you forward syslog from rsyslogd, you might encounter a mismatch between UTC and CET. To fix this use this configuration:
# Provide a propper timeformat to fix the UTC/CET mismatch $template forward_template,"<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" *.* @@127.0.0.1:5514;forward_template
how can you store the data after container resetart? what volume should the configs/files be saved to to survive
Guys, since the commenting function here is aweful, plz use the github issue system for errors. Much better to keep track there.
First let me thank you for this docker file.. I been working on trying to understand the mechanics. So far I have been able to get the container up and running. But I have nto been able to get data to appear on the kibana interface... I have added the elastick head and bigdesk.
./bin/plugin -install mobz/elasticsearch-head
./bin/plugin -install lukas-vlcek/bigdesk
and I am maybe not understanding the process, Can someone help me understand how logstash gets data to elasticsearch..
Moin @garystaffird & @nicovillanueva (sorry for the delay, kinda missed out on it),
I updated the README.md on github, but it did not trickle down to dockerhub. :(
I updated it by hand. I already provide an
Please check this out, the previous stated option were a bit outdated.
I've created a
docker-compose.yaml file to create the elk container using
elk: image: qnib/elk:latest ports: - "8200:80" volumes: - "/dev/null:/dev/null" - "/dev/urandom:/dev/urandom" - "/dev/random:/dev/random" - "/dev/full:/dev/full" - "/dev/zero:/dev/zero" privileged: true hostname: elk environment: - "HTTPPORT=8200"
However, I'm getting an error similar to some other users have reported, earlier:
[root@elk opt]# supervisorctl status consul RUNNING pid 15, uptime 0:35:59 diamond RUNNING pid 14, uptime 0:35:59 elasticsearch FATAL Exited too quickly (process log may have details) kibana RUNNING pid 23, uptime 0:35:59 logging FATAL Exited too quickly (process log may have details) logstash RUNNING pid 13747, uptime 0:00:20 logstash_watchdog RUNNING pid 11, uptime 0:35:59 nginx RUNNING pid 32, uptime 0:35:59 sshd RUNNING pid 13, uptime 0:35:59 statsd RUNNING pid 12, uptime 0:35:59 syslog-ng RUNNING pid 18, uptime 0:35:59 watchpsutil RUNNING pid 13844, uptime 0:00:05 [root@elk opt]#
Any troubleshooting suggestions?
Hey qnib, I'm trying out your Dockerfile to check out the ELK stack.
I'm possibly doing something wrong, but I ran into a few issues:
- After setting all exports and running the run command, the following error came up: Error response from daemon: Could not get container for carbon
- Once in the Kibana's UI, a Connection Error showed, saying: “Possibility #2: You are running Elasticsearch 1.4 or higher” and, well, I couldn't do much more… Is there more to do than just running the above commands?
Hey @jonathanbowker, sorry - I have never used Beanstalk. Why ain't you start the container on a standard AWS EC2 instance - or even start it via ECS?
@qnib - Please can you help. I am new to Docker and trying to install this on AWS via Beanstalk. Please could you send me a link to the "Application Version" version file. Many thanks.
Hey @abferm, I figured that an interactive startup makes not much sense in times where we got
docker exec, since you can introduce your shell after the fact. But you might got a point here. Do you reckon to explain this in a bit more detail? Thx for the feedback?
@qnib What happened to your explanations of how to run interactively vs in the background?