bigdatateam/hysh repository overview

Instead of IMAGE in the command below use the following value for obtaining the pure “hysh” image:

• hysh:py3

To run container you need to install docker and then execute:

docker run -it --rm -v YYYYYY:/home/jovyan/dir_on_host -p XXXX:80 IMAGE

-it allows to see the information from the container

--rm means that the container will be deleted after exiting/terminating

-v means that the files from the Jupyter of the container will be transferred to the local file system directory YYYYYY

YYYYYY - an absolute local path, for example, for Linux $(pwd)/bigdata, $(pwd) - the path to the current directory

-p allows to connect to the NGINX of the container on 127.0.0.1:XXXX or localhost:XXXX, XXXX - a port number of the local system

Addresses provided by NGINX:

• http://127.0.0.1:XXXX/jup⁠ (Jupyter Notebook)
• http://127.0.0.1:XXXX/hdfs_ui⁠
• http://127.0.0.1:XXXX/hdfs_node⁠
• http://127.0.0.1:XXXX/yarn_ui⁠
• http://127.0.0.1:XXXX/yarn_node⁠
• http://127.0.0.1:XXXX/yarn_jobhistory⁠
• http://127.0.0.1:XXXX/spark_ui⁠
• http://127.0.0.1:XXXX/spark_jobhistory⁠

Alternative way is to bind the services ports to the host port instead of binding the NGINX port:

• -p AAAA:8888 — Jupyter Notebook
• -p BBBB:50070 — HDFS UI
• -p CCCC:50075 — HDFS node
• -p DDDD:8088 — YARN UI
• -p EEEE:8042 — YARN node
• -p FFFF:19888 — YARN jobhistory server
• -p GGGG:4040 — Spark UI
• -p HHHH:18080 — Spark jobhistory server

Note: if you choose the alternative way and run container on a remote system, -e REMOTE_HOSTNAME=RRRRRR is useful in order to fix the services redirections.

--rm and -v YYYYYY:/home/jovyan/dir_on_host can be safely removed if it is not needed.

For more options see the "docker run" documentation page⁠.