Public | Automated Build

Last pushed: 2 months ago
Short Description
Runs as a daemoset in k8s to send node logs to humio
Full Description

k8s-humio - Ship Logs from Kubernetes to Humio

Contains components for shipping logs from kubernetes clusters to humio.

Overview

Here, fluentd is used to forward application- and _host-_level logs from each kubernetes worker node to humio server. This extends the standard setup here from fluentd for log forwarding in kubernetes.

Node-level Forwarding

In fluentd/docker-image a docker image is defined which specifies how to forward to humio (with other settings, like log sources reused from the base image). Kubernetes manifests are defined in fluentd/k8s: a daemonset will deploy fluentd pods across every worker node inside the kube-system namespace, and each pod will read the humio ingest token from the fluentd-humio-ingest-token secret.

As per the normal setup, fluentd output is buffered, and uses tls for nice log confidentiality.

Setup

  1. Pre-requisites:
    • Kubernetes cluster
    • User authorized to administrate via kubectl
    • Default service account with read privileges to API server for use by the kubernetes metadata filter plugin. This should be present by default in the kube-system namespace (even in kubernetes 1.6 with RBAC enabled)
  2. Setup your dataspace in humio and create an ingest token
  3. Base64 encode your token by running printf 'TOKEN' | base64 and update fluentd/k8s/fluentd-humio-ingest-token-secret.yaml with the value
  4. Create your resources in kubernetes: kubectl apply -f fluentd/k8s/
  5. Logs start appearing in humio!
Docker Pull Command
Owner
benjvi
Source Repository

Comments (0)