Short Description
Spark 2.0.1 including Java 8 and Python 3.5
Full Description
To start the master:
docker run -d --name sparkmaster --net=host -e ROLE=master nicomak/spark:2.0.1
To start slaves:
docker run -d --name sparkslave --net=host -e ROLE=slave -e MASTER_URL=spark://[master_host]:7077 nicomak/spark:2.0.1
More info
- The
--net=host
network mode if prefered because Spark opens a lot of random ports. - Any Spark environment variables can be defined in the container run command through
-e
options.
Reference
More details about this image on my blog: http://blog.ditullio.fr/2016/11/09/docker-spark-analyse-cassandra-data/
Docker Pull Command
Owner
nicomak