HyperLoom is a platform for defining and executing workflow pipelines in a distributed environment. HyperLoom is a highly scalable framework that is able to efficiently execute millions of interconnected tasks on hundreds of computational nodes.
- Optimized dynamic scheduling with low overhead.
- In-memory data storage with a direct access over the network with a low I/O footprint.
- Direct worker-to-worker data transfer for low server overhead.
- Third party application support.
- Data-location aware scheduling reducing inter-node network traffic.
- C++ core with a Python client enabling high performance available through a simple API.
- High scalability and native HPC support.
- BSD license.