Latest vanilla llama.cpp for gfx906
100K+
Minimal llama.cpp runtime image for AMD (gfx906) GPUs.
llama-server, llama-bench, llama-cli| Tag | ROCm | Base OS | GPU |
|---|---|---|---|
rocm-633 | 6.3.3 | Ubuntu 20.04 | Radeon Instinct MI50, Radeon Instinct MI60, Radeon VII, Radeon Pro VII |
rocm-727 | 7.2.7 | Ubuntu 24.04 | Radeon Instinct MI50, Radeon Instinct MI60, Radeon VII, Radeon Pro VII |
docker run -it --device=/dev/kfd --device=/dev/dri --group-add video \
-v /models:/models mxxm/llama.cpp-gfx906:rocm-633
Drops into bash. Run commands directly:
llama-server -m /models/model.gguf -ngl 99 --host 0.0.0.0 --port 8080
llama-bench -m /models/model.gguf -ngl 99
llama-cli -m /models/model.gguf -ngl 99
services:
llama:
image: mxxm/llama.cpp:rocm-633
devices:
- /dev/kfd
- /dev/dri
group_add:
- video
volumes:
- /models:/models
ports:
- "8080:8080"
entrypoint: ["llama-server"]
command: ["-m", "/models/model.gguf", "-ngl", "99", "--host", "0.0.0.0"]
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 5s
retries: 3
Content type
Image
Digest
sha256:57291c177…
Size
967.2 MB
Last updated
3 days ago
docker pull mxxm/llama-cpp-gfx906:rocm-727