openeuler/mooncake repository overview

⁠Quick reference

• The official Mooncake docker image.

• Maintained by: openEuler CloudNative SIG⁠.

• Where to get help: openEuler CloudNative SIG⁠, openEuler⁠.

⁠Mooncake | openEuler

Current Mooncake docker images are built on the openEuler⁠. This repository is free to use and exempted from per-user rate limits.

Mooncake is a high-performance transfer engine for distributed AI inference workloads, developed by KVCache.AI. It provides a RDMA/TCP-based data plane for efficient KV cache transfer between disaggregated prefill and decode instances in large language model (LLM) serving systems. Mooncake is designed for distributed storage and high-speed data movement in cloud-native AI infrastructure.

Learn more about Mooncake on GitHub⁠⁠.

⁠Supported tags and respective Dockerfile links

The tag of each mooncake docker image is consist of the version of mooncake and the version of basic image. The details are as follows

⁠Usage

In this usage, users can select the corresponding {Tag} based on their requirements.

• Pull the openeuler/mooncake image from docker

  docker pull openeuler/mooncake:{Tag}
  

  Copy

• Run the transfer engine benchmark (sender side)

  docker run --rm --network host openeuler/mooncake:{Tag} \
      transfer_engine_bench --mode=initiator \
      --metadata_server=<metadata-ip>:2379 \
      --local_server_name=<local-ip>:12345
  

  Copy

• Run the transfer engine benchmark (receiver side)

  docker run --rm --network host openeuler/mooncake:{Tag} \
      transfer_engine_bench --mode=target \
      --metadata_server=<metadata-ip>:2379 \
      --local_server_name=<local-ip>:12346
  

  Copy

• Validate the transfer engine

  docker run --rm --network host openeuler/mooncake:{Tag} \
      transfer_engine_validator --metadata_server=<metadata-ip>:2379
  

  Copy

• View available tools

  Tool	Description
  transfer_engine_bench	Benchmark the transfer engine throughput and latency
  transfer_engine_validator	Validate transfer correctness between two nodes
  transfer_engine_bench_with_notify	Benchmark with notification-based completion

⁠Question and answering

If you have any questions or want to use some special features, please submit an issue or a pull request on openeuler-docker-images⁠.