Public | Automated Build

Last pushed: 5 months ago
Short Description
Dockerfile providing ELK services (Elasticsearch/Logstash/Kibana)
Full Description

docker-elk

Dockerfile creating ELK services (Elasticsearch/Logstash/Kibana)

Please use at least 2GB of RAM for this one... See #16

It's available on hub.docker.com, just pull it:
docker pull qnib/elk

Parts

It will

  • connects with consul, if available
  • start sshd
  • start logstash
  • start diamond
  • start StatsD
  • start elasticsearch
  • start nginx (kibana3)
  • start kibana4

How to use kibana3 and kibana4 could be explored within this 'hello world' blog post.

Within QNIBTerminal

To get the most out of it a carbon container might be added, but this will impose the question whether to go even further and distribute all the services.

Known issues

Time mismatch in rsyslog

If you forward syslog from rsyslogd, you might encounter a mismatch between UTC and CET. To fix this use this configuration:

# Provide a propper timeformat to fix the UTC/CET mismatch
$template forward_template,"<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
*.* @@127.0.0.1:5514;forward_template
Docker Pull Command
Owner
qnib
Source Repository

Comments (37)
cardins2u
2 years ago

how can you store the data after container resetart? what volume should the configs/files be saved to to survive

qnib
2 years ago

Guys, since the commenting function here is aweful, plz use the github issue system for errors. Much better to keep track there.

llacayo
2 years ago

First let me thank you for this docker file.. I been working on trying to understand the mechanics. So far I have been able to get the container up and running. But I have nto been able to get data to appear on the kibana interface... I have added the elastick head and bigdesk.

./bin/plugin -install mobz/elasticsearch-head
./bin/plugin -install lukas-vlcek/bigdesk

and I am maybe not understanding the process, Can someone help me understand how logstash gets data to elasticsearch..

Thanks

qnib
2 years ago

Moin @garystaffird & @nicovillanueva (sorry for the delay, kinda missed out on it),

I updated the README.md on github, but it did not trickle down to dockerhub. :(
I updated it by hand. I already provide an docker-compose.yml file.

Please check this out, the previous stated option were a bit outdated.

Cheers
Christian

garystafford
2 years ago

I've created a docker-compose.yaml file to create the elk container using docker-compose.

elk:  
image: qnib/elk:latest  
ports:  
  - "8200:80"  
volumes:  
 - "/dev/null:/dev/null"  
 - "/dev/urandom:/dev/urandom"  
 - "/dev/random:/dev/random"  
 - "/dev/full:/dev/full"  
 - "/dev/zero:/dev/zero"  
privileged: true  
hostname: elk  
environment:  
- "HTTPPORT=8200"  

However, I'm getting an error similar to some other users have reported, earlier:

[root@elk opt]# supervisorctl status  
consul                           RUNNING    pid 15, uptime 0:35:59  
diamond                          RUNNING    pid 14, uptime 0:35:59  
elasticsearch                    FATAL      Exited too quickly (process log may have details)  
kibana                           RUNNING    pid 23, uptime 0:35:59  
logging                          FATAL      Exited too quickly (process log may have details)  
logstash                         RUNNING    pid 13747, uptime 0:00:20  
logstash_watchdog                RUNNING    pid 11, uptime 0:35:59  
nginx                            RUNNING    pid 32, uptime 0:35:59  
sshd                             RUNNING    pid 13, uptime 0:35:59  
statsd                           RUNNING    pid 12, uptime 0:35:59  
syslog-ng                        RUNNING    pid 18, uptime 0:35:59  
watchpsutil                      RUNNING    pid 13844, uptime 0:00:05  
[root@elk opt]# 

Any troubleshooting suggestions?

nicovillanueva
2 years ago

Hey qnib, I'm trying out your Dockerfile to check out the ELK stack.
I'm possibly doing something wrong, but I ran into a few issues:

  • After setting all exports and running the run command, the following error came up: Error response from daemon: Could not get container for carbon
  • Once in the Kibana's UI, a Connection Error showed, saying: “Possibility #2: You are running Elasticsearch 1.4 or higher” and, well, I couldn't do much more… Is there more to do than just running the above commands?
qnib
2 years ago

Hey @jonathanbowker, sorry - I have never used Beanstalk. Why ain't you start the container on a standard AWS EC2 instance - or even start it via ECS?

jonathanbowker
2 years ago

@qnib - Please can you help. I am new to Docker and trying to install this on AWS via Beanstalk. Please could you send me a link to the "Application Version" version file. Many thanks.

qnib
2 years ago

Hey @abferm, I figured that an interactive startup makes not much sense in times where we got docker exec, since you can introduce your shell after the fact. But you might got a point here. Do you reckon to explain this in a bit more detail? Thx for the feedback?

abferm
2 years ago

@qnib What happened to your explanations of how to run interactively vs in the background?