Public | Automated Build

Last pushed: a year ago
Short Description
Spark cluster image for dev
Full Description


for Mac OS

  • cassandra
  • cassandra-2
  • postgres
  • zookeeper
  • kafka
  • spark-master
  • spark-worker1
  • spark-worker2
  • spark-worker3

All sparks nodes share one path "/jar" on outside Docker it's in "/tmp/docker/spark-allnodes"
Services save their state in /tmp/docker dir. Remove it if you want to clear all data.

0. Prepare (need pseudo real ip)

sudo ifconfig lo0 alias up

1. Examples

Run full infrastructure

docker-compose -f up 

Zookeeper with Kafka

docker-compose -f up zookeeper kafka

Postgres with Cassandra one node

docker-compose -f up postgress cassandra

Postgres with Cassandra two nodes

docker-compose -f up postgress cassandra cassandra-2

Spark Cluster with 3 workers

docker-compose -f up spark-master spark-worker-1 spark-worker-2 spark-worker-3

Docker Pull Command
Source Repository