kvcacheai/mooncake repository overview

Mooncake is an open-source KVCache-centric infrastructure project for large-scale LLM serving. It provides a disaggregated serving architecture, high-performance data transfer, and distributed KVCache management to help AI infrastructure teams improve inference throughput, resource utilization, and reliability for production-scale workloads.

Source code: https://github.com/kvcache-ai/mooncake⁠

Documentation: https://kvcache-ai.github.io/Mooncake/⁠