Sign inSign up

bogs/fluentd-aws-elasticsearch

By bogs

•Updated almost 11 years ago

Container for automating and dynamically allowing AWS Elasticsearch to be setup at runtime.

Image
1

1.8K

bogs/fluentd-aws-elasticsearch repository overview

⁠Fluentd AWS Elasticsearch

Setup and initializes the Fluentd AWS Elasticsearch plugin in a container using plugin: fluent-plugin-aws-elasticsearch-service⁠ at version 0.1.4 and tiller⁠ at version 0.7.4

It is also using the latest Debian Jessie linux instance as the base image.

⁠Configuration

The container dynamically generates the inbound and outbound configuration options using a json file.

Both the input and output default property name and values found in the fluentd DSL input⁠ and output⁠ can be used. To further extend functionality the output includes the elasticsearch⁠ and elasticsearch-aws⁠.

⁠Example
{
  "sources": [
    {
      "type": "forward",
      "port": "12345",
      "bind": "0.0.0.0"
    },
    {
      "type": "syslog",
      "port": "12345",
      "bind": "0.0.0.0",
      "tag": "syslog",
      "protocol_type": "tcp",
      "format": "json"
    },
    {
      "type": "tcp",
      "tag": "tcp.events",
      "format": "json",
      "port": "12345",
      "bind": "0.0.0.0"
    }
  ],
  "stores": [
    {
      "type": "aws-elasticsearch-service",
      "index_name": "indexname",
      "flush_interval": "5s",
      "buffer_type": "memory",
      "buffer_queue_limit": "64",
      "buffer_chunk_limit": "8m",
      "logstash_format": "true",
      "endpoint": {
        "region": "<region>",
        "url": "<url>",
        "access_key_id": "<accesskeyid>",
        "secret_access_key": "<secret_access_key>"
      }
    }
  ]
}
⁠Docker run

To setup you need to pass it a json or file in the envirionment variable using -e tiller_json='$(cat config.json)'

⁠AWS Elasticsearch Gotcha

You need to have both the access_key_id and secret_access_key as the bug in the library exist⁠.

Tag summary

Content type

Image

Digest

sha256:2b083aa5c…

Size

242.9 MB

Last updated

almost 11 years ago

docker pull bogs/fluentd-aws-elasticsearch