truespirit/vllm-rocm10-gfx950 repository overview

⁠truespirit/vllm-rocm10-gfx950

If you are able to squeeze something usefull out of it - let me know by now i cant beat rocm/vllm-dev:nightly_main_20260211 for serving https://huggingface.co/truespirit/Ornith-1.5-397B-MXFP4⁠

Unofficial community-built vLLM container for AMD Instinct™ MI355X (gfx950), based on the AMD ROCm™ PyTorch container.

This image is maintained by a private individual for experimentation and community use. It is not an official AMD, PyTorch, vLLM, FlashAttention, AITER, or Quark image and is not affiliated with, sponsored by, or endorsed by those projects or their respective maintainers.

⁠Image

docker pull truespirit/vllm-rocm10-gfx950:latest

⁠Main components

The image is approximately 57 GB.

⁠Purpose

This image exists to provide a reproducible vLLM userspace for gfx950, built around the ROCm 10 PyTorch base image.

Several components are built from source specifically for the target architecture.

The image is intentionally specialized for MI355X / gfx950. It should not be considered a general-purpose ROCm or vLLM image.

No model weights are included.

⁠Running

The host must have a working ROCm-compatible AMD GPU driver and expose the required GPU devices to Docker.

Example interactive start:

docker run --rm -it \
  --device=/dev/kfd \
  --device=/dev/dri \
  --group-add video \
  --ipc=host \
  --shm-size 8G \
  truespirit/vllm-rocm10-gfx950:latest

The default command is:

/bin/bash

On an MI355X system, run the included sanity check first:

verify-on-mi355x

For an initial vLLM test, using eager execution is recommended:

vllm serve <model> --enforce-eager

Some AITER/FlyDSL kernels are compiled by JIT during the first GPU warmup, so the first run can take longer than later runs.

⁠Hugging Face cache

The image uses:

HF_HOME=/models/hf

A host cache can for example be mounted there:

docker run --rm -it \
  --device=/dev/kfd \
  --device=/dev/dri \
  --group-add video \
  --ipc=host \
  --shm-size 8G \
  -v "$HOME/.cache/huggingface:/models/hf" \
  truespirit/vllm-rocm10-gfx950:latest

Model files downloaded or mounted by the user are not part of this image and remain subject to their respective model licenses and terms.

⁠Important limitations

• Built specifically for gfx950.
• Primary intended hardware is AMD Instinct MI355X.
• This is an experimental/community build.
• It has not been certified by AMD or the upstream projects.
• Compatibility with other ROCm versions, GPUs, drivers, models, or host configurations is not guaranteed.
• Some kernels may be JIT-compiled during first use.
• No commercial support, SLA, or maintenance commitment is provided.

⁠Third-party software and licenses

This container contains and redistributes third-party open-source software.

Nothing in this Docker Hub repository relicenses those components. Each component remains subject to its own copyright notices, license terms, attribution requirements, and other applicable conditions.

Major components include, among others:

This table is only a convenience summary. The container contains additional direct and transitive dependencies that may use different licenses or include additional notices.

Users and redistributors should review the license and notice files supplied by the respective upstream projects and by the software installed inside the image.

Where an upstream license requires preservation of copyright notices, license text, attribution, NOTICE files, or similar information when redistributing binary software, those requirements continue to apply.

⁠No affiliation or endorsement

Names such as AMD, AMD Instinct, AMD ROCm, PyTorch, vLLM, FlashAttention, AITER, Quark, Docker, Hugging Face, and other product or project names are used only to identify compatibility and included software.

This project does not claim ownership of third-party trademarks, project names, software, or logos and does not imply endorsement by their owners.

AMD ROCm™ and AMD Instinct™ are trademarks of Advanced Micro Devices, Inc.

Other product names and trademarks belong to their respective owners.

⁠Disclaimer

This image is provided on an “AS IS” and “AS AVAILABLE” basis, without warranties or guarantees of any kind.

Use of this image is at your own risk. The maintainer makes no guarantee regarding fitness for a particular purpose, availability, performance, security, correctness, compatibility, or absence of defects.

This image may contain experimental software and very large compiled binaries. Review the image and its dependencies before using it in security-sensitive, production, regulated, or otherwise critical environments.

The maintainer is not responsible for model licenses, datasets, prompts, generated output, or other content that users choose to process with the image.

⁠Licensing note

The fact that this image is published by a private individual or used for personal, research, or noncommercial purposes does not replace or override the licenses of the software contained in the image.

Before redistributing or modifying the image, users should independently verify that their intended use complies with all applicable upstream licenses and terms.

This README provides a best-effort description for informational purposes and is not legal advice.