Public Repository

Last pushed: a year ago
Short Description
Tag v10: Spark 2.0.2
Full Description

docker service create --name spark-master \
--constraint 'node.hostname==master' \
-p 8080:8080 -p 7077:7077 -p 4040:4040 \
vetional/spark:v10 \
/start-master.sh

docker service create --name spark-worker --mode global \
--constraint 'node.hostname!=master' \
-p 8081:8081 \
vetional/spark:v10 \
/start-worker.sh spark://<SPARK_MASTER_IP>:7077

Modify number of workers as required using:
docker service scale spark-worker=N

docker exec -dit <SPARK_MASTER_CONTAINER_ID> \
/usr/spark-2.0.2/bin/spark-submit --master spark://<SPARK_MASTER_IP>:7077 \
examples/src/main/python/als.py

Docker Pull Command
Owner
vetional