juiceboxdocks/vllm-runtime repository overview

⁠vLLM Runtime

Reproducible vLLM runtime for NVIDIA Ada GPUs on RunPod.

• vLLM: 0.21.0+cu129
• PyTorch: CUDA 12.9 build
• Base: runpod/pytorch:2.4.0-py3.11-cuda12.4.1-devel-ubuntu22.04

Pull a pinned image tag:

docker pull juiceboxdocks/vllm-runtime:0.21.0-cu129-ada

Model weights and Hugging Face credentials are intentionally not included.