Public Repository

Last pushed: 2 years ago
Short Description
This image is based on Spark 1.2.0 and Hadoop 2.6.0 version.
Full Description

Along with Spark, and Yarn , this image also has important python machine libraries installed on the system core like numpy, scipy, and scikit-learn. The tag with mmm shows the code for Market Mixed Models implementation

The image works for both yarn-cluster and yarn-client modes

Estimating Pi (yarn-cluster mode):

execute the the following command which should write the "Pi is roughly 3.1418" into the logs

spark-submit --class org.apache.spark.examples.SparkPi --master yarn-cluster --driver-memory 1g --executor-memory 1g --executor-cores 1 $SPARK_HOME/lib/spark-examples-1.2.0-hadoop2.4.0.jar

Estimating Pi (yarn-client mode):

execute the the following command which should print the "Pi is roughly 3.1418" to the screen

spark-submit --class org.apache.spark.examples.SparkPi --master yarn-client --driver-memory 1g --executor-memory 1g --executor-cores 1 $SPARK_HOME/lib/spark-examples-1.2.0-hadoop2.4.0.jar

Docker Pull Command
Owner
swatisoni

Comments (0)