voipmonitor/vllm repository overview

⁠voipmonitor/vllm:test-cu132

vLLM inference stack for RTX PRO 6000 Blackwell (SM120a) with CUDA 13.2.

⁠Stack

• Ubuntu 24.04 + CUDA 13.2.0 + cuDNN
• PyTorch nightly (cu132)
• vLLM (source build)
• FlashInfer (git main)
• SGLang (source, editable)
• nvidia-cutlass-dsl 4.4.2 (with CUDA 13.1 NVVM/ptxas fix)

⁠Key fix: nvidia-cutlass-dsl NVVM/ptxas version

nvidia-cutlass-dsl[cu13] ships two sub-packages that both write _cutlass_ir.so:

• libs-base (mandatory) — NVVM/ptxas 12.9 (can't compile _mma.block_scale for SM120)
• libs-cu13 (optional extra) — NVVM/ptxas 13.1 (works)

pip installs libs-base after libs-cu13, overwriting the CUDA 13 binary. Fix: pip install --force-reinstall --no-deps nvidia-cutlass-dsl-libs-cu13 as last step.

⁠Dockerfile

See full Dockerfile in the source repository.