fraserpricee/bonsai-vllm repository overview

Serves Bonsai 2 27B⁠, Prism ML's ternary Qwen3.8-27B, on vLLM with custom CUDA kernels. Unofficial; not affiliated with Prism ML. Built and tested for the RTX PRO 6000 Blackwell only.

docker run --rm --gpus all --ipc=host -p 8000:8000 -v bonsai:/cache fraserpricee/bonsai-vllm:20260918

Source, details and issues: https://github.com/fraserprice/bonsai-vllm⁠

Weights: https://huggingface.co/fraserprice/Ternary-Bonsai-2-27B-vllm⁠

Apache 2.0. Created using Bonsai by Prism ML; built from Qwen3.8-27B; image built on vLLM.