bigdatauniversity/spark2 repository overview

⁠Apache Spark Image for the Spark Fundamentals II Course

This Docker image should be used for creating environments for conducting hands-on labs for the Spark Fundamentals II⁠ course on the www.bigdatauniversity.com⁠. You can use it to create Spark environment on your own laptop/desktop or on one of the supported public clouds.

This docker image contained pre-deployed Apache Hadoop with Spark and Zeppelin notebooks.

• Download Docker Toolbox for Mac ⁠ or Docker Toolbox for Windows(patched,v1.8.1d)⁠, and install on your laptop

  Instructions for Windows⁠ Instructions for Mac⁠

⁠How to use this image ?

⁠Kitematic (GUI)

1. Start Kitematic in Docker folder

2. type bigdatauniversity in the search box to filter the Docker Hub catalog to Big Data University provided images

3. Click on Create button on the spark image to create Docker container using this image

⁠Docker Quickstart Terminal (CLI)

For Mac -- "Applications -> Docker -> Docker Quickstart Terminal" For Windows -- "Start -> Program -> Docker -> Docker Quickstart Terminal".

Then run the below steps within this terminal.

1. Pull (download) this Docker image Run this command in your terminal window:

docker pull bigdatauniversity/spark2

1. Start Spark container

docker run -it --name bdu_spark2 -P -p 4040:4040 -p 4041:4041 -p 8080:8080 -p 8081:8081 bigdatauniversity/spark2:latest /etc/bootstrap.sh -bash

1. Note

• How to restart and attach to the container

If you exit from Docker Container, you can always restart and attach to it later by running the below:

docker start  bdu_spark2 
docker attach bdu_spark2 

• Start a new command in a running container

docker exec -it bdu_spark2 <command>

⁠Supported tags

• latest
• 1.3.1

The supported tags stands for version of Spark.

⁠Supported Docker versions

• This image is officially supported on Docker version 1.6.0.
• Support for older versions (down to 1.0) is provided on a best-effort basis.

⁠Community Support

• Zhong Yu (Leo) Wu ( [email protected]⁠ )
• Emerging Technology Team ( [email protected]⁠ ), IBM Analytics Platform