Public | Automated Build

Last pushed: 3 years ago
Short Description
Short description is empty for this repo.
Full Description

Apache Spark Shell node image for Docker

Based on prodriguezdefino/sparkbase image.

This image is intended to run alongside with a cluster formed by one master node image and as many worker node images as needed.

The default CMD will load up a bash inside the container, running docker run -it --name=shell -h shell -p 4040:4040 --link=master:master --rm=true prodriguezdefino/sparkshell will spawn the container (it should be assured that master, shell and worker must be reacheable between each other, maybe docker links can be used to do that).

In order to start using the interactive shell, once inside the container, we can fire up $RUN_SPARK_SHELL master (where master is the url of the Spark master node). This will configure the shell to connect to our master node (linked by docker at spawn time) and get ready to run calculations across the cluster.

Docker Pull Command