stingrayzboy/pyspark repository overview

This is a pyspark image that i created to help learn and teach pyspark to new students. just clone the repo and run

docker container run -it -v $(pwd):/app -p 80:8888 stingrayzboy/pyspark bash

You will enter the bash terminal of the alpine linux which contains pyspark

for Windows machine running Docker you need to provide the proper location in the -v option example:

docker container run -it -v D:\Sites\:/app -p 80:8888 stingrayzboy/pyspark bash

This comes with Jupyter Notebook preinstalled so you can run jupyter notebook with the following commands

jupyter notebook --ip=0.0.0.0 --allow-root

This allows you to forward the communication to the docker host browser

The URL generated by the console needs to be modified.

Rather than using the specified URL just hit localhost on your browser and enter the token as a param

P.S. We are facing some resolution issues in the import pyspark therefore I have added findspark module import findspark which needs to be imported first to it you can use the findspark.init('/spark') to get it working