Public | Automated Build

Last pushed: a year ago
Short Description
Open source data collection, parsing, and enrichment pipeline
Full Description

docker-logstash

Logstash is a flexible, open source data collection, parsing, and enrichment pipeline. With connectors to common infrastructure for easy integration, Logstash is designed to efficiently process a growing list of log, event, and unstructured data sources for distribution into a variety of outputs, including Elasticsearch.

More details on the Logstash product can be found at the elastic website at https://www.elastic.co/products/logstash

This Docker build builds on top of a Java image to provide a working Logstash instance to connect to your Elasticsearch instance.

Under the most basic usage you will make sure it is operating in the same network (i.e. same machine) as the cluster it will be connected to. It is recommened you use restart on-failure, prevent swaping, and limit RAM usage of the container to just over 4GiB.

docker run -d --restart=on-failure --memory="6442450944" --memory-swap="-1" solnetcloud/logstash:latest

Please note that if you need a Lumberjack interface you will need to provide an SSL certificate --lm-ssl-crt and --lm-ssl-key. Lumberjack in this configuration requires JSON input and will be available on port 8888.

NOTICE: As this build is designed to run using multicast it is recommend you utilise Weave, or bridge your docker0 on a common subnet in order to facilate the multicast. If you require unicast please override the configuration file

WARNING: DO NOT OUTPUT LOGS TO SYSLOG FOR THIS CONTAINER. Especially if a stdout logging is enabled. This could create a feedback loop where events are processed multiple times.

WARNING: Please note that as syslog is a very muddy term this input only supports RFC3164 syslog with some small modifications. The date format is allowed to be RFC3164 style or ISO8601. Otherwise the rest of RFC3164 must be obeyed. If you do not use RFC3164, DO NOT USE this input.

usage: entry [-h] [--stdout] [--hash-key [HASH_KEY]] [--use-sha512]                                                                                                                                                                          
            [--es-cluster-name [ES_CLUSTER_NAME]]                                                                                                                                                                                           
            [--es-node-name [ES_NODE_NAME]] [--lm-ssl-crt [LM_SSL_CRT]]                                                                                                                                                                     
            [--lm-ssl-key [LM_SSL_KEY]] [--lm-type [LM_TYPE]]                                                                                                                                                                               
            [--lm-tags [LM_TAGS [LM_TAGS ...]]] [--ignore-match-errors]                                                                                                                                                                     

optional arguments:                                                                                                                                                                                                                          
-h, --help            show this help message and exit                                                                                                                                                                                      
--stdout              Also output logs processed to stdout for debug (Not                                                                                                                                                                  
                        Recommend)

hashing:                                                                                                                                                                                                                                     
Arguments specific to hashing                                                                                                                                                                                                              

--hash-key [HASH_KEY], -k [HASH_KEY]                                                                                                                                                                                                       
                        The hash key used by Logstash. (Make sure it is
                        consistent when running multiple instances) (Default
                        "docker-logstash")
--use-sha512, -5      By default this container uses SHA256 for hashing,
                        override and use SHA512.

elasticsearch:
Arguments specific to connecting to Elasticsearch

--es-cluster-name [ES_CLUSTER_NAME], -c [ES_CLUSTER_NAME]
                        The name of the cluster the Elasticsearch Instance
                        should connect to
--es-node-name [ES_NODE_NAME], -n [ES_NODE_NAME]
                        The node name this logstash node should use when
                        connecting to Elasticsearch

lumberjack:
Arguments for when you want to use Lumberjack input

--lm-ssl-crt [LM_SSL_CRT], -R [LM_SSL_CRT]
                        Certificate for SSL termination, under the /ls-
                        data/ssl/ volume
--lm-ssl-key [LM_SSL_KEY], -K [LM_SSL_KEY]
                        SSL Key for SSL termination, under the /ls-data/ssl/
                        volume
--lm-type [LM_TYPE], -T [LM_TYPE]
                        If you want the lumberjack messages to have a specific
                        type, state it here
--lm-tags [LM_TAGS [LM_TAGS ...]], -t [LM_TAGS [LM_TAGS ...]]
                        If you want the lumberjack messages to have tags
                        applied, state them here
--ignore-match-errors
                        Ignore SSL certificate match errors. (Not recommended)
Docker Pull Command
Owner
solnetcloud
Source Repository

Comments (0)