omegang/fluentd-logzio-kubernetes repository overview

⁠Collecting Docker Log Files with Fluentd and Logz.io⁠

This directory contains the source files needed to make a Docker image that collects Docker container log files using Fluentd⁠ and sends them to your Logz.io⁠ account.

This image is designed to be used as a daemonset⁠ in a Kubernetes⁠ cluster.

⁠Usage

Sign in to your logz.io account. Fluentd connection details will be available at https://app.logz.io/#/dashboard/data-sources/Fluentd⁠

⁠Customize daemonset.json

⁠Environment variables

• Replace _YOUR_ENDPOINT_URL_ with your logz.io access endpoint, such as https://listener.logz.io:8071⁠.
• Replace _YOUR_TOKEN_ with your logz.io account token.
• Replace _YOUR_LOG_TYPE_ with the type of logs you are shipping. We are using this as a tag to identify the k8s cluster source of the logs.
• Replace _LOG_LEVEL_ with the requested fluentd log level. See the logging documentation⁠ for more details. Also uncomment this line⁠.

The logz.io endpoint URL is ${_YOUR_ENDPOINT_URL_}?token=${_YOUR_TOKEN_}&type=${_YOUR_LOG_TYPE_}

⁠Mount points

• The host's /var/log path will be used for fluentd symlinks to the container log files. Needs to be writable.
• The host's $DOCKER_ROOT/containers path ($DOCKER_ROOT=/var/lib/docker in our case, your mileage may vary) is where the docker container log files are placed. This is where the symlinks from /var/log/containers point to. Needs to be readable. See caveat below, change according to your docker root path.

⁠Docker tag

• Replace _YOUR_DOCKER_IMAGE_TAG_ with the docker image tag you will build from this repo.

⁠Build and deploy

• docker build -t ${_YOUR_DOCKER_IMAGE_TAG_} .
• docker push ${_YOUR_DOCKER_IMAGE_TAG_}
• kubectl create -f daemonset.json --validate=false (disable validation over readOnly key in one of the volumes)

⁠Caveats

• Because of an issue with mounting symlinked directories⁠, you may need to change the mount points in daemonset.json to include your kubelet's actual docker root.

⁠Origin

This repo is based on https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch/fluentd-es-image⁠

It was first created by the folks of DoiT International⁠

Used and published by Snyk⁠ for the benefit of Logz.io⁠ users