Public Repository

Last pushed: 7 months ago
Short Description
Spark 2.0.1 including Java 8 and Python 3.5
Full Description

To start the master:

docker run -d --name sparkmaster --net=host -e ROLE=master nicomak/spark:2.0.1

To start slaves:

docker run -d --name sparkslave --net=host -e ROLE=slave -e MASTER_URL=spark://[master_host]:7077 nicomak/spark:2.0.1

More info

  • The --net=host network mode if prefered because Spark opens a lot of random ports.
  • Any Spark environment variables can be defined in the container run command through -e options.

Reference

More details about this image on my blog: http://blog.ditullio.fr/2016/11/09/docker-spark-analyse-cassandra-data/

Docker Pull Command
Owner
nicomak

Comments (0)