Sign inSign up

tairangroup/spark

By tairangroup

Updated over 8 years ago

SPARK ENVIRONMENT

Image
0

258

tairangroup/spark repository overview

== Spark Environment

Based tairangroup/hadoop:latest , Depend gudaoxuri/scala_2.11:latest

=== Features

  • SSH server & client ( Port: 22 User: root Password: 123456 )
  • Hadoop 2.7.x
  • Spark 2.3.x

=== Examples

Start Spark:

docker run --name spark -h spark -P -e KEEP=true -d gudaoxuri/spark

Operation:

[source,shell]

Hadoop Home:/opt/spark/

example:

run-example org.apache.spark.examples.SparkPi

spark-submit
--class org.apache.spark.examples.SparkPi
--master spark://spark:7077
--deploy-mode cluster
--driver-memory 512M
--executor-memory 512M
/opt/spark/examples/jars/spark-examples_2.11-2.2.1.jar
1000


=== Environments

|=== | Env | Default Value | Remark

| TZ | Asia/Shanghai | | KEEP | false | true = always run |===

=== Expose Ports

|=== | Port | Remark

| 22 | SSH Port

| 4040 | REST API | 7077 | SPARK_MASTER_PORT | 8080 | SPARK_MASTER_WEBUI_PORT | 8081 | SPARK_WORKER_WEBUI_PORT |===

Tag summary

Content type

Image

Digest

Size

951.8 MB

Last updated

over 8 years ago

docker pull tairangroup/spark