Public Repository

Last pushed: 3 months ago
Short Description
Single-node deployment of DataTorrent’s Real Time Streaming (RTS) Platform.
Full Description

DataTorrent RTS test sandbox, powered by Apache Apex, Apache Hadoop.

This repository contains a ready to use DataTorrent RTS test sandbox docker image in which datatorrent RTS is preinstalled and running. All one need to do is launch this docker image and start playing with DataTorrent RTS.

Basic

Run RTS docker container

This will create and start a docker container from the docker image.
Note: You can change the RTS version number based on the docker image you are using. We are using the latest for example.

Running in interactive mode
docker run -it --name=RTS-sandbox -p 9090:9090 -p 50070:50070 -p 8088:8088 datatorrent/rts:latest  
Running in detached mode

Note: It takes few seconds to start the dtgateway.

docker run -d --name=RTS-sandbox -p 9090:9090 -p 50070:50070 -p 8089:8088 datatorrent/rts:latest
Start dtcli after launching docker container
dtadmin@176d473e7eba:~$ dtcli
Apex CLI 3.5.0 16.09.2016 @ 01:43:00 PDT rev: 30bb100 branch: refs/tags/v3.5.0
apex> 

Advanced

Linux user information

Username: dtadmin
Password: dtadmin

NOTE: User "dtadmin" is added in sudoers list and is provided with root privileges.

Run docker container with host directory mounted

This will create and start a docker container and docker images while mounting local filesystem directory as a mount point inside docker container.

docker run -it --name=RTS-sandbox -v /local/path/to/mount:/mount_location -p 9090:9090 -p 50070:50070 -p 8088:8088 datatorrent/rts:latest
Start already created docker container
docker start -i RTS-sandbox
DtManage, Hadoop, YARN WebUI from local machine

Following command will map dtmanage, yarn and hadoop ports exposed inside docker container to be mapped to ports of host machine.

docker run -it --name=RTS-sandbox -p 9090:9090 -p 50070:50070 -p 8088:8088 datatorrent/rts:latest

After docker has started, one can point host machine's browser to below URLs:
localhost:9090 : DtManage WebUI
localhost:8088 : yarn WebUI
localhost:50070 : hadoop WebUI

NOTE: If any of these ports on host machine are address already in use, then change the port to another unused port.
For example: if port 8088 is already in use on your host machine then use another unused port example: 8089

docker run -it --name=RTS-sandbox -p 9090:9090 -p 50070:50070 -p 8089:8088 datatorrent/rts:latest

In this case you can access yarn WebUI using localhost:8089

Getting Help

The docker deployment approach is a great way to learn about DataTorrent RTS, stream data processing, and Apache Apex. Please remember that the docker sandbox image is not intended or supported for production use.

If you need help or would like to share feedback, please visit our Community Forum. We’d love to hear from you.

Please also see DataTorrent’s documentation and our website for further information, as well as the license agreement associated with the docker sandbox image.

Docker Pull Command
Owner
datatorrent