louisremi/llama-server-rdna-gfx1150 repository overview

⁠llama-server-rdna-gfx1150

A self-contained llama.cpp server image for AMD RDNA 3.5 Strix Point APUs (gfx1150), built from the stew675/llama-cpp-rdna-boosts⁠ fork with 16 RDNA-specific performance patches applied.

⁠Why this image exists

The prebuilt ghcr.io/stew675/llama-cpp-rdna-boosts images (rocm-7.2 / 7.14 / 10.0) only compile GPU kernels for gfx1100;gfx1151;gfx1200;gfx1201. They do not include gfx1150, so on Strix Point APUs (e.g. the Radeon 890M iGPU in the Ryzen AI 9 HX 370) every kernel launch fails with:

ROCm error: device kernel image is invalid

This image rebuilds the patched source with -DAMDGPU_TARGETS=gfx1150, so it actually runs on gfx1150 silicon.

⁠Target hardware

• Strix Point (Ryzen AI 300 series: Ryzen AI 9 HX 370/375, Ryzen AI 7 350, etc.) — reports as gfx1150
• Any AMD APU whose iGPU reports as gfx1150 to ROCm

⁠What's inside

• Base: ghcr.io/stew675/llama-cpp-rdna-boosts:rocm-10.0 (ROCm 10.0, amdclang/hipcc)
• Source: upstream ggml-org/llama.cpp at commit ebbb18522
• Patches: all 16 RDNA-boosts patches (blocks 00–15) applied on top
• Build: cmake -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1150 -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF -DGGML_RPC=OFF
• Binary: llama-server (the OpenAI-compatible HTTP server)
• Entrypoint: /src/llama.cpp/build/bin/llama-server
• Env: LD_LIBRARY_PATH=/opt/rocm/lib

⁠Usage

⁠Single model

docker run --rm \
  --device /dev/dri --device /dev/kfd --group-add video \
  -v /path/to/models:/models:ro \
  -p 8080:8080 \
  --entrypoint /bin/bash \
  louisremi/llama-server-rdna-gfx1150:rocm-10.0-gfx1150 \
  -c 'export LD_LIBRARY_PATH=/opt/rocm/lib; \
      /src/llama.cpp/build/bin/llama-server \
        -m /models/your-model.gguf \
        -c 32768 -ngl 999 --port 8080'

⁠Multi-model router (auto-loads on demand)

docker run -d \
  --name llama-rdna \
  --device /dev/dri --device /dev/kfd --group-add video \
  -v /path/to/models:/models:ro \
  -p 8190:8080 \
  --entrypoint /bin/bash \
  louisremi/llama-server-rdna-gfx1150:rocm-10.0-gfx1150 \
  -c 'export LD_LIBRARY_PATH=/opt/rocm/lib; \
      /src/llama.cpp/build/bin/llama-server \
        --models-dir /models --jinja --host 0.0.0.0 \
        --port 8080 -ngl 999 -c 32768 --api-key YOUR_KEY'

Then query the OpenAI-compatible API:

curl http://localhost:8190/v1/completions \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"Sharp-Spark-X2.5-4B-Q4_K_XL","prompt":"Hello","max_tokens":32}'

⁠Benchmarks (Ryzen AI 9 HX 370, Radeon 890M iGPU, 47 GB VRAM)

(Compared to stock Vulkan: Sharp-Spark 16k pp=543, tg=26.3 — the RDNA build wins on prefill, is roughly tied on decode.)

⁠Rebuilding

A self-contained Dockerfile is available in the compose directory — or rebuild from scratch:

# 1. Clone the fork for the patches
git clone --depth 1 https://github.com/stew675/llama-cpp-rdna-boosts /tmp/fork

# 2. Clone upstream at the fork point
git clone --filter=blob:none https://github.com/ggml-org/llama.cpp
cd llama.cpp && git checkout ebbb18522

# 3. Apply the 16 patches
for p in /tmp/fork/patches/*.patch; do git apply "$p"; done

# 4. Build for gfx1150 (in a ROCm 10.0 container with cmake installed)
cmake -B build -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1150 \
  -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF -DGGML_RPC=OFF
cmake --build build --target llama-server -j $(nproc)

⁠Tags

• rocm-10.0-gfx1150 — ROCm 10.0 base, built for gfx1150 (current)
• latest — same as rocm-10.0-gfx1150

⁠License

The RDNA-boosts patches are distributed under the terms of the stew675/llama-cpp-rdna-boosts⁠ repository. Upstream llama.cpp is MIT-licensed.