Docker Logstash Integration
- any Docker log files
- whatever log files you configure logstash-forwarder to ship within a container (just put a config at
/etc/logstash-forwarder.conf, only the
filessection gets evaluated while
networksection is globally configured).
I wasn't too happy with existing possibilities and while I know that the Docker team is working on a solution, this scratches my itch right now.
Also I didn't see an obvious way to extend docker-gen to handle generic in container templates.
Besides that, how much reason do you need to play with Go & Docker? ;-)
How it works:
docker-logstash-forwarder listens to Docker events and continually restarts a logstash-forwarder instance, after refreshing its configuration, every
laziness seconds after a new event was received (to avoid unnecessary restarts - configurable via
-laziness flag - defaults to 5 seconds).
For every running container the docker log file is added and it is checked if a logstash-forwarder config exists within the container at
If an in container specific config exists, the path of all files will be expanded to be valid within the logstash-forwarder container before adding them to the global configuration.
This requires the following (in container defaults in brackets):
- read-only access to the directory containing your docker data (
- connection to Docker (
- connection to Logstash (
Read-only access to Docker data:
Mount the directory containing your Docker data into the containers
/var/lib/docker - i.e. run the container with
-v /var/lib/docker:/var/lib/docker:ro (assuming your Docker files are stored in
/var/lib/docker on the host).
Connection with Docker:
For communication with Docker the following endpoints are evaluated:
- whatever is passed via the
-dockercommand line flag
It is suggested to use the later - as in run the container with
Behind the screens fsouza/go-dockerclient is used for communication with Docker.
Connection with Logstash:
For communication with Logstash the following endpoints are evaluated:
- whatever is passed via the
-logstashcommand line flag
This allows you to
docker -link your Logstash instance to the containers
logstash-forwarder authentication can be managed in the following ways:
- specify a custom config pointing to some imported volume containing the required cert & key via the
-configflag (only the
networksection is evaluated)
- make your keys available bellow
TL;DR / Quickstart:
$ docker pull digitalwonderland/logstash-forwarder $ docker run -d --name logstash-forwarder -v /var/lib/docker:/var/lib/docker:ro -v /var/run/docker.sock:/var/run/docker.sock --link logstash:logstash --volumes-from logstash digitalwonderland/logstash-forwarder
If you start from scratch / use Vagrant / are on a Mac: just clone this repository and run
vagrant up. This gives you a VM based on CoreOS (which is awesome btw) running those 3 containers & Kibana listening to localhost:8888 (Docker listens to localhost:2375).
- the container is pretty big (>500MB). This is because everything is build from source since sharing your whole docker directory requires quite some trust. This might change once I found a way to reliable integrate some build service with docker hub so binaries get build on commit and only after that a docker build is triggered which includes the just built binaries.
- docker-logstash-forwarder must be run as root until Docker provides configurable ownership of shared volumes, because
/var/lib/dockeris owned by root on the host and mounted read only, so a non root user can not read from it.
Last but not least it probably should be mentioned, that this is the first time I wrote any go code (a few days, after work), so any 'Duh' pointers are greatly appreciated.
Pull Requests welcome :)