Public Repository

Last pushed: 6 months ago
Short Description
fluentd docker image with ruby 2.2.x and support for both elasticsearch and kinesis
Full Description

Overview

fluentd docker image with ruby 2.2.x and support for both elasticsearch and kinesis

Configuration

Create a fluentd.conf file following fluentd's configuration file
format. Add any input that you wish to and add the kinesis output
(see example file in this repository).

Installed plugins

The following plugins are installed in the docker image:

  1. fluent-plugin-ec2-metadata
  2. fluent-plugin-hostname
  3. fluent-plugin-retag
  4. fluent-plugin-kinesis
  5. fluent-plugin-kinesis-aggregation
  6. fluent-plugin-elasticsearch
  7. fluent-plugin-record-modifier
  8. fluent-plugin-multi-format-parser

Here is an example configuration to use them. Note that this uses
the old 'match/retag' approach; ideally you should use filter plugins
like record-modifier (see fluentd.conf), but unfortunately there
is no filter equivalent for ec2metadata yet.

<match syslog>
  type hostname
  key_name ec2_hostname
  add_prefix hostname
</match>
<match hostname.syslog>
  type ec2_metadata
  output_tag ec2.${tag}
  <record>
    ec2_instance_id   ${instance_id}
    ec2_instance_type ${instance_type}
    ec2_az            ${availability_zone}
    service_id        my-service-uuid
    env               my-environment
  </record>
</match>
<match ec2.hostname.**>
  type retag
  remove_prefix ec2.hostname
</match>

Run

Mount the volumes where your logs are if needed and the path to the
fluentd configuration file. Pass aws credentials via environment
variables:

docker run --ulimit nofile=65536:65536 -p 24224 -p 24220 -p 24230 \
        -v /var/log:/fluentd/log -v `pwd`:/etc/fluent \
        atlassianlabs/fluentd:0.5.1
Docker Pull Command
Owner
atlassianlabs