zenth815/mx-llama-rocm10-gfx906 repository overview

⁠mx-llama.cpp + ROCm 10, for AMD gfx906 (Instinct MI50/MI60, Radeon VII)

mx-llama.cpp⁠ (a llama.cpp fork) built against a custom ROCm 10 distribution for gfx906 — a GPU architecture that ROCm 10 doesn't officially support upstream. Combines mx-llama.cpp's own work with gfx906-specific fixes and optimizations ported in from several community forks, verified on real dual-MI50 hardware.

⁠What's in it beyond stock mx-llama.cpp

• RoPE __sincosf fusion and a custom gfx906 SGEMM kernel
• A wave64 GCN MMQ tuning table, plus an MI50-measured tile override for Q5_K
• A crash fix for SOLVE_TRI on gfx906's rocBLAS (affects Gated Delta Network models: Qwen3.5, Qwen3-Next, Kimi Linear)
• Turbo KV cache compression (--cache-type-k/v turbo2/turbo3/turbo4) — roughly 3.3x more KV cache capacity at a generation-speed cost, confirmed working across a 2-GPU --split-mode layer setup including cross-turn cache read-back
• Confirmed on real hardware: Q4_K/Q5_K/Q6_K/Q8_0 weight repack and fused MoE up/gate mat-vec firing correctly across both GPUs

See GFX906-ROCM10.md⁠ for the full list and provenance.

⁠Run it

docker run -d --name mx-llama-rocm10 \
  --device=/dev/kfd --device=/dev/dri \
  --ipc=host --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
  -p 8080:8080 -v /path/to/models:/models \
  -e HIP_VISIBLE_DEVICES=0,1 \
  zenth815/mx-llama-rocm10-gfx906:latest \
  --server -m /models/model.gguf -ngl 99 -fa 1 -sm layer --host 0.0.0.0 --port 8080

⁠Built on

zenth815/rocm10-gfx906:sdk⁠ — the custom gfx906 ROCm 10 build this image compiles against.

⁠Source

• github.com/Wizard815/mx-llama.cpp-Rocm10⁠
• ROCm build: github.com/Wizard815/TheRock-gfx906⁠