Public | Automated Build

Last pushed: 2 years ago
Short Description
Spark 1.6.1 cluster prepared with sparkling water-1.6.3
Full Description

Docker container for sparkling water stand alone cluster

This repository contains a set of scripts and configuration files to run a Apache Spark standalone cluster together with Sparkling water from Docker container.

To run master execute:


To run worker execute:


You can run multiple workers. Every worker would be able to find master by it's container name "spark-master".

To run spark shell against this cluster execute:


To run Run Sparkling Shell:


Sparkling Shell accepts common Spark Shell arguments.
For example, to increase memory allocated by each executor, use the spark.executor.memory parameter: bin/sparkling-shell --conf "spark.executor.memory=4g"

Initialize H2OContext:

import org.apache.spark.h2o._
import org.apache.spark.examples.h2o._
val hc = H2OContext.getOrCreate(sc)
Docker Pull Command