Public | Automated Build

Last pushed: 2 years ago
Short Description
Dockerized version of distributed dask for computing on a cluster.
Full Description

Distributed dask on Docker

I was working on this as a research alternative to fighting the Spark / MapReduce workflows in highly iterative algorithms run on a cluster.

Of course, I quickly discovered Oliver Grisel had already done something very similar and arguably more robust. Still, different use-cases and simple stubborn curiosity compelled me to keep working on my own.


This is a [HIGHLY EXPERIMENTAL] containerized implementation of distributed dask. If you're really curious, check out Matthew Rocklin's blog posts on the topic.

More details to come as the implementation progresses.

Docker Pull Command
Source Repository