imvexxed/nadcs_node repository overview

⁠NaDCS

Not a Distributed Computing System, a rapidly deployable and highly scaleable distributed computing system for C# code using Docker.

⁠First Beta

NaDCS has entered into it's first Beta, the program functions, albeit sometimes roughly and lacking in functions with bugs to share, but it works.

⁠Planned Features

• Scheduler Web Management Interface
• Ability to unload tasks from nodes
• Better dynamic exception handling
• Option for network encryption/compression
• More library extensability

⁠So What Does It Do?

NaDCS [nɑ dɪks] intends to be a highly scaleable and rapidly deployable distributed computing system. First I'd like to establish some terms you'll see in the code a lot.

• Node: A singular end-point that works to compute the requests submitted
• Task: A singular request that will be distributed to a node to be computed
• Scheduler: The central host that orchestrates all the Nodes and ballances the Tasks between them.
• Client: The program or logic that submits Tasks to the Scheduler

The system intends to take on a star topology (atleast logically), and is simple in concept. A client, or any piece of arbitrary software is built but needs to execute a lot of complex code that takes too much time for it to complete on it's own, the solution is obvious but talk is easy.

This is where NaDCS comes in.

NaDCS's Node and Scheduler system will never need to be modified by the average user or developer, both are pre-built containerized images ready to go with only a few command line arguments seperating you from a working cluster of computing nodes. The only work that needs to be done is in the taskable code. Any code to be distributed to the nodes needs to be transplanted into a library that implements the ITaskable interface. And thats really it. After that it's a breeze to orchestrate tasks with the Scheduler from the Client using the provided library, the taskable code will be sent to the Scheduler and will then be distributed to the Nodes and dynamically loaded and prepared for Tasking. Each node will be notified and passed contextual parameters unique to that specific Task when the Scheduler decides to task the node. You only have to write the code once. (If you've used my NLC code before this is effectively the inverse of the standard call flow, instead of the client calling functions on the server, the server calls functions in the client.)

⁠Notes

• For any coming from NLC, the transition to .Net Core 2.0 was painless, every thing could just be dropped in and work, there was no need to re-write anything.... almost. Of course there was one thing, Eliptic Curve Diffie Hellman which is used to establish the keys in the client and server's handshake doesn't exist in .Net Core 2.0. And after doing some research it seems it's being perpetually post-poned⁠ by the corefx team.