Public Repository

Last pushed: 8 months ago
Short Description
Short description is empty for this repo.
Full Description


the version no. such as 2.0.1 means we are using spark 2.0 , and the the last number is for our version number


  1. start spark-slave with ${SPARK_HOME}/bin/spark-class org.apache.spark.deploy.worker.Worker $MASTER_URL

which enables to run spark process at foreground , that will benefit us from running spark in docker (kubernetes).
Once there is something wrong with the process , such as can't connect to master, it will shut down , and kubernetes rc will help us recover the pod

Docker Pull Command