rocm/vllm-dev
vLLM-dev is a weekly dev stream toolkit and library for large language model (LLM) inference and serving. These containers are a quick way to run or try vLLM on AMD Instinct™ accelerators. For building or extensions for vLLM-dev, please see the Build Dockerfiles.
Running Containers
docker run -it --rm \
--ipc=host \
--network=host \
--privileged \
--cap-add=CAP_SYS_ADMIN \
--device=/dev/kfd \
--device=/dev/dri \
--device=/dev/mem \
--group-add render \
--cap-add=SYS_PTRACE \
--security-opt seccomp=unconfined \
rocm/vllm-dev:main
Documentation
Please see here for full documentation on getting started, setting up the environment, and more.
docker pull rocm/vllm-dev