Sign inSign up

1gog/spark

By 1gog

•Updated over 6 years ago

Image
0

1.3K

1gog/spark repository overview

spark 2.4.5

url: https://apache-mirror.rbc.ru/pub/apache/spark/spark-2.4.5/spark-2.4.5-bin-hadoop2.7.tgz⁠

Build image:

  1. download spark tgz unpack in first image and them copy to second images, so second image not contain tgz archive example:
FROM centos:7 
RUN curl http://spark.tgz
RUN tar -xzf spark.tgz -C /opt

FROM centos:7 
RUN mkdir -p /opt/spark-2.4.5 && ln -s /opt/spark-2.4.5 /opt/spark2
COPT --from=0 /opt/spark-2.4.5/ /opt/spar2/
  1. second form wrote a label FROM image:tag AS label_for_tmp_image example:
FROM centos:7 AS spark2
RUN curl http://spark.tgz
RUN tar -xzf spark.tgz -C /opt

FROM centos:7 
RUN mkdir -p /opt/spark-2.4.5 && ln -s /opt/spark-2.4.5 /opt/spark2
COPT --from=spark2 /opt/spark-2.4.5/ /opt/spar2/
  1. You are can use one more use case, copy file from other image
COPY --from=nginx:latest /etc/nginx/nginx.conf /nginx.conf

How to run docker image

docker run -it --rm --name spark2 -p 4040:4040  1gog/spark:2.4.5 bin/spark-shell 

docker run -it --rm --name spark2 -p 4040:4040  1gog/spark:2.4.5 bin/spark-submit

docker run -it --rm --name spark2 -p 4040:4040  1gog/spark:2.4.5 bin/pyspark

Tag summary

Content type

Image

Digest

Size

439.6 MB

Last updated

almost 7 years ago

docker pull 1gog/spark