renien/spark-stand-alone repository overview

~/projects/personal/docker-spark-stand-alone$ sudo docker-compose up
Creating spark-worker-1 ... done
Creating spark-master   ... done
Attaching to spark-worker-1, spark-master
spark-worker-1    | 2021-01-17 04:29:42,210 INFO worker.Worker: Started daemon with process name: 1@35ef8be86a36
spark-master      | 2021-01-17 04:29:42,210 INFO master.Master: Started daemon with process name: 1@master

~/projects/personal/docker-spark-stand-alone$ sudo docker exec -it spark-master /bin/bash
[sudo] password for renien: 
root@master:/usr/spark-2.4.7# spark-shell 
2021-01-17 04:38:20,713 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Spark context Web UI available at http://localhost:4040
Spark context available as 'sc' (master = spark://master:7077, app id = app-20210117043827-0000).
Spark session available as 'spark'.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.4.7
      /_/
         
Using Scala version 2.11.12 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_271)
Type in expressions to have them evaluated.
Type :help for more information.

scala>