dataspree/dlds-training repository overview

⁠Data Spree Deep Learning DS Training Worker

The Deep Learning DS Training Worker is the essential part of Data Spree's AI Platform that is used to train Deep Neural Networks. The Worker connects to the Deep Learning DS Server from where it downloads the training parameters and data. Training jobs get assigned to the Worker which typically executes these jobs in the background. Learn more on the Deep Learning DS plarform here: https://docs.data-spree.com/deep-learning-ds⁠

The Worker requires an Nvidia GPU and the Nvidia Container Toolkit (https://github.com/NVIDIA/nvidia-docker⁠). We recommend a GPU with at least 8GB of memory.

Create a container for the Worker in the background mapping the relevant directories:

docker run -d --name dlds-worker --runtime nvidia -p 6714:6714 -v $(pwd)/training:/root/.dlds -v $(pwd)/training:/data dataspree/dlds-training dlds-worker --dataset_directory /data

Once you've started the Worker, you can connect to it using the dlds-client which is a Python package including a command-line interface to configure the worker:

pip install dlds-client
dlds worker setup

In case you favor a graphical user interface, you can download the Deep Learning DS Training Worker Hub from here: https://www.data-spree.com/resources/releases⁠

To start or stop the container, you can use the following commands:

docker start dlds-worker
docker stop dlds-worker

View the logs of the running container via:

docker logs dlds-worker