lamastex/hsbase repository overview

⁠hsBase

A docker image containing Apache Hadoop and Apache Spark. It is preconfigured for using hdfs and Yarn in pseudo-distributed mode, i.e. launching a master and a worker on the same node.

⁠Usage

To start a container use the command docker run -it lamastex/hsbase. It will automatically start a shell inside the container. From here you can either start hdfs or Yarn manually or use the included script start.hs which will automatically format the hdfs namenode and start a namenode and datanode.

⁠Ports

The following ports are exposed by the dockerfile:

• hdfs webUI: 50070
• Yarn webUI: 8088
• Spark master webUI: 7070
• Spark worker webUI: 8081

Note that the Spark master webUI port is not the default 8080 to avoid collision with Zeppelin which also uses port 8080 by default.