Public | Automated Build

Last pushed: 2 years ago
Short Description
Short description is empty for this repo.
Full Description

docker-centos-spark

A Dockerfile that produces a Centos-based Docker image that will run a standalone pre-build version of ApacheSpark.

Image Creation

This example creates the image with the tag internavenue/centos-spark, but you can change this to use your own username.

$ docker build -t="internavenue/centos-spark" .

Alternately, you can run the following if you have GNU Make installed...

$ make

You can also specify a custom docker username like so:

$ make DOCKER_USER=internavenue

Usage

Standalone single-node use

sudo docker run -t -i -p internavenue/centos-spark:centos7

Spark is by default placed at /opt/spark

You can run a test (estimate Pi number) with the following command within the container

/opt/spark/bin/run-example SparkPi 10

Standalone cluster mode

  • Spin-up the master container
sudo sh start-master.sh
  • Spin-up workers:
sudo sh start-worker.sh
  • To launch a Spark shell against the master:
sudo sh spark-shell.sh
Docker Pull Command
Owner
internavenue
Source Repository

Comments (0)