Public Repository

Last pushed: 6 months ago
Short Description
Kulcloud PRISM ELK stack
Full Description

What is Kulcloud PRISM ELK

This documents how to use the kulcloud ELK Docker image and sFlow, which provides a convenient centralised traffic log server and log management web, by packaging Elasticsearch, Logstash, and Kibana, collectively known as ELK and sFlow collector

Installation

Install Docker, either using a native package (Linux) or wrapped in a virtual machine (Windows, OS X – e.g. using Boot2Docker or Vagrant). To pull this image from the Docker registry, open a shell prompt and enter:
$ sudo docker pull kongseokhwan/kulcloud_elk:E1L1K4

Note – The last image using the Elasticsearch 1.x and Logstash 1.x branches can be pulled using sudo docker pull kongseokhwan/kulcloud_elk:E1L1K4. The versions of the ELK components in that image are: Elasticsearch 1.7.3, Logstash 1.5.5, and Kibana 4.1.2.

Usage

######Usage Run the container from the image with the following command:
$ sudo docker run --net=host -p 5601:5601 -p 9200:9200 -p 5044:5044 -p 5000:5000 -it --name elk kongseokhwan/kulcloud_elk:E1L1K4

  • 5601 (Kibana web interface).
  • 9200 (Elasticsearch JSON interface).
  • 5044 (Logstash Beats interface, receives logs from Beats such as Filebeat – see the Forwarding logs section below).
  • 5000 (Logstash Lumberjack interface, receives logs from Logstash forwarders – see the Forwarding logs section below).

Note - The image also exposes Elasticsearch's transport interface on port 9300. Use the -p 9300:9300 option with the docker command above to publish it.

Access Kibana's web interface by browsing to http://<your-host>:5601, where <your-host> is the hostname or IP address of the host Docker is running on (see note), e.g. localhost if running a local native version of Docker, or the IP address of the virtual machine if running a VM-hosted version of Docker (see note).
Starting a logstash to get a flow log informations from sFlow logstash forwarder Open a shell prompt in the container and type (replacing <container-name> with the name of the container, e.g. elk in the example above):
$ sudo docker exec -it <container-name> /bin/bash
At the prompt, enter:
# /opt/logstash/bin/logstash –f log_file.conf 
Configure log_flie.conf according your configurtion
input {
 udp { 
   port => 5000                                                              #logstash server listening Port
   type => "sflow"
   codec => "json"
  }
}
filter {
  json {
      source => "message"
  }
}
output {
     elasticsearch {
          hosts => ["localhost"]
     }
}

sFlow logstash

sFlow

Tiny sflow collector and parser script based on eventmachine. It listens for sflow v5 samples, parses them and sends it to logstash.

Installation

Install Docker, either using a native package (Linux) or wrapped in a virtual machine (Windows, OS X – e.g. using Boot2Docker or Vagrant).
To pull this image from the Docker registry, open a shell prompt and enter:
$ docker pull kongseokhwan/sflow_logstash_forwarder
Run the container from the image with the following command:
$ sudo docker run --net=host -it --name sflow_logstash_forwarder kongseokhwan/sflow_logstash_forwarder:v1.1

Usage

Starting a sFlow logstash forwarder
Open a shell prompt in the container and type (replacing <container-name> with the name of the container, e.g. sflow_logstash in the example above):
$ sudo docker exec -it <container-name> /bin/bash
Configure your logstash endpoint
Configure log_flie.conf according your configurtion
# cd /home/kulcloud/sflow
# vim ./etc/config.yaml
And then execute:
# bundle exec ./bin/sflow.rb
Docker Pull Command
Owner
kongseokhwan

Comments (0)