fundisoftware/taw-elastic repository overview

⁠Operations Analytics for z/OS in Elastic using IBM Transaction Analysis Workbench for z/OS

The taw-elastic Docker⁠ image demonstrates using the Elastic Stack⁠ to analyze operational data from IBM z/OS.

The image contains the Elastic Stack configured with:

• Kibana dashboards for analyzing operational data from various z/OS subsystems, such as CICS, IMS, and DB2
• Sample data for the dashboards

The sample data was forwarded from z/OS to Elastic by IBM Transaction Analysis Workbench for z/OS⁠ (colloquially known as Workbench or TAW).

This image provides a quick and easy way—single-command installation, if you already have Docker—to try the dashboards in a self-contained “sandbox” environment, with sample data.

⁠Watch the video

For an introduction to the dashboards, watch the YouTube video IBM Transaction Analysis Workbench for z/OS - Operations analytics dashboards in Splunk and Elastic⁠.

⁠Getting started

The following procedure gets the taw-elastic image from the Docker Hub website, and then starts a container based on that image. This might take several minutes, depending on your connection to the web, and the computer on which you are running Docker (your Docker host). You can then use a web browser to view the sample dashboards in the container.

If your organization uses a different interface than the Docker command-line interface to manage Docker containers, then use the details from the docker run command provided here to specify equivalent details in that other interface.

To start using the demo:

1. Get a Docker host. Either:

• Install Docker⁠ on your personal computer, or
• Contact your organization's software support for details of an existing Docker host

If you install Docker, follow the installation instructions on the Docker website, including the steps to verify installation. In particular, ensure that you can successfully run the Docker “hello world” example. 2. Dedicate the following minimum computing resources to Docker:

• CPUs: 2
• Memory: 4096 MB (if you cannot dedicate this much memory to Docker, dedicate as much as you can: try progressively smaller amounts)

Tip: If you are using Docker for Windows, open the Windows system tray, right-click the Docker icon, select Settings ► Advanced, and then adjust these settings. 3. Check that your Docker host has enough free disk space for the image: at least 1 GB. 4. Check that your Docker host virtual memory settings meet the Elasticsearch requirements⁠. 5. On your Docker host, open a command prompt and enter the following command:

docker run -d -p 15044:5044 -p 15601:5601 -p 19200:9200 --name taw-elastic fundisoftware/taw-elastic:v0.0

The -p command options map ports inside the container to the following ports on your Docker host:

• Logstash listens on port 15044 for incoming data. To add your own data to the container, use TAW to stream JSON Lines to this port.
• Kibana is on port 15601.
• The Elasticsearch API is on port 19200.

If these port numbers clash with existing ports on your Docker host, feel free to use different port numbers. For details, see the Docker command reference documentation⁠. 6. Wait: for the Docker image to download, for the container to start, and then for the container to initialize Elastic with the supplied dashboards and data. Depending on your connection the web, the Docker image might take several minutes to download. After that, the container might take another minute to initialize, depending on your Docker host. 7. Browse to the following URL:

http://localhost:15601/app/kibana#/dashboards?_g=(time:(from:'2017-08-14T06:09:26.289Z',to:'2017-08-14T07:20:51.671Z'))

• Replace localhost in the URL with the name of the Docker host.
• The time parameter in the URL specifies the time range of the sample data, so that you do not have to specify this range to Kibana yourself.
• We recommend the Chrome web browser.

Your web browser should display a list of Kibana dashboards. Click a dashboard and begin exploring the data. For details on using Kibana, see the Kibana User's Guide⁠.

The first time the container starts, it loads data into Elastic. Kibana allows you to view dashboards while the data is still loading, so you might see partially filled charts, such as vertical time-based bar charts with little or no data showing on the right-hand side. If this happens, wait a little longer, and then refresh the page in your browser (for example, press F5).

If your web browser does not display a list of Kibana dashboards, or you click a dashboard and the dashboard contains no data, or you experience some other problem, see “Troubleshooting”.

⁠Troubleshooting and support

Before contacting Fundi Software support, please ensure that you can successfully run the Docker “hello world” example described in the Docker documentation. Fundi offers support for this Docker image; we do not offer support for general Docker installation and usage issues.

If you experience a problem using these dashboards, enter the following command on your Docker host, and save the command output:

docker logs taw-elastic

Then contact your local Docker expert or Fundi Software support⁠.

⁠Tips for new Docker users

⁠Stopping the container

The docker run command in the “Getting started” procedure specifies a -d option that starts a container in “detached” mode rather than the default foreground mode.

In detached mode, the container continues running after you close the command shell that you used to start it, so you don't need to keep that command shell open.

To stop the container, enter the following command:

docker stop taw-elastic

Shutting down your computer stops Docker and the container. When you reboot your computer and Docker restarts, the container will not restart automatically: you need to restart it. To change this behavior, see the Docker documentation on restart policies.

⁠Restarting the container

To restart the container after stopping it or after rebooting your computer, enter the following command:

docker restart taw-elastic

⁠Other Docker commands

To list containers, both running and stopped, use the docker ps -a command.

To remove a container, use the docker rm command.

For more information about Docker commands, see the Docker documentation⁠.

⁠About the Elastic Stack in the image

The taw-elastic Docker image is based on the sebp/elk Docker image. The sebp/elk image provides the Elastic Stack (version 5.5.0). taw-elastic adds Kibana dashboards and sample data.

For information about the sebp/elk Docker image, including how volumes can be used to persist data, see the documentation⁠, Docker Hub page⁠, and GitHub repository⁠.

⁠Loading data into the container

In the container, Logstash is configured to listen for TAW-format JSON Lines over TCP, on port 5044. You can use TAW to stream data to the container, or you can use a utility such as netcat to transfer a file of JSON Lines.

For details on forwarding your own logs for use with these sample dashboards, see the example JCL⁠.

⁠Controlling the loading of sample data and Kibana objects

When you run a container for the first time, you can set environment variables to control the loading of sample data and Kibana objects into the container. (The Kibana objects define the sample dashboards.) These variables are in addition to the environment variables provided by the base sebp/elk image.

• INSTALL_SAMPLES

Defaults to 1, which loads the sample data and Kibana objects (sample searches, visualizations, dashboards, and the default index pattern).

To disable all loading, set to 0 and omit all other variables.

• INSTALL_SAMPLE_DATA

Defaults to the value of INSTALL_SAMPLES.

To skip loading the sample data, set to 0.

To load the sample data, set to 1.

• INSTALL_SAMPLE_OBJECTS

Defaults to the value of INSTALL_SAMPLES.

To skip loading Kibana objects, set to 0.

To load Kibana objects, set to 1.

If you load Kibana objects but not the sample data then Kibana will not function correctly until an index matching the default index pattern fuw-* exists in Elasticsearch.

You can set these environment variables using the -e option of the docker run command.

For example, if you want to use the sample dashboards, but you want to omit the sample data because you plan to forward your own logs to the container, set INSTALL_SAMPLE_OBJECTS=1 and INSTALL_SAMPLE_DATA=0:

docker run -d -p 15044:5044 -p 15601:5601 -p 19200:9200 --name taw-elastic-mydata -e INSTALL_SAMPLE_DATA=0 -e INSTALL_SAMPLE_OBJECTS=1 fundisoftware/taw-elastic:v0.0

⁠Who we are

Fundi Software⁠ develops IBM Transaction Analysis Workbench for z/OS, and developed this image, including the Kibana dashboards.

⁠Copyright

© Copyright Fundi Software 2017, 2018.