elephantscale/es-training repository overview

⁠ES-Training Docker Image

⁠License

This work is licensed under Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)⁠

To obtain a commercial license for this work, or to customize this for your needs, please contact us at [email protected]⁠

⁠About

One image to rule them all

This image is created with the goal of making learning Machine Learning / Deep Learning / Big Data stack easier.

It has most commonly used software installed and configured, so students can start working with the technologies, rather than wasting a lot of time setting up.

⁠Software Installed

⁠Latest version: prod2 (recommended)

• based on Ubuntu 20.04
• Java 11 + Scala 2.12 + SBT 1.4 + Maven 3.6.3 + Eclipse 2020-09
• Spark 3.0.1
• Zookeeper 3.4.12 + Zookeeper Web
• Kafka 2.5.0 + Kafka Manager 2.0.0
• Anaconda Python 3.8
• Deep Learning: Tensorflow 2.3.1 + Pytorch 1.4.0
• Machine Learning: scikit-learn-0.23

⁠Step 1: Need to have latest docker platform installed

Get it from docker.com⁠

⁠Step 2 : Download latest docker image

Versions

• prod2 version : recommended for most users (based on ubuntu 20.04)
• prod or r-020 version : based on ubuntu 18.04
• latest or dev version : experimental

    $  docker pull  elephantscale/es-training:prod2
    $  docker images
    # you should see 'elephantscale/es-training:prod2'

⁠Step 3 : Download Data files

• You can download our dataset from here⁠

• Unzip data file into ~/data directory.
  This directory will be mounted by docker under /data directory

⁠Step 4: Run Script

• (public) shell script⁠
• (public) windows powershell script⁠

Download this script and save it as run-docker.sh

⁠Step 5: Running the 'es-training' Docker

#  Now go to the project directory
$  cd /path/to/my_project

#  Start docker from this project directory.
#  This directory will be mounted in ~/dev directory in docker
#  ElephantScale/projects @ host --> ~/dev  within container

#  run it (adjust your path)
$  bash run-docker.sh

# or

# run with custom password
$  bash run-docker.sh -p MY_PASSWORD

Go to http://localhost/⁠ to access the page

⁠Step 6 : Default Password

For Jupyter and SSH and noVNC the password is : bingobob123

⁠Step 7: Accessing files

⁠Dev Notes

See README-dev.md⁠