underlabsai/decisor-sglang repository overview

⁠decisor-sglang

Patched SGLang inference runtime for decisor-4b⁠, a typed-decision model by Under Labs⁠.

Based on SGLang v0.5.20, with a fused-checkpoint loader fix and a logprobs guard. Model weights are downloaded separately on first startup.

⁠Quickstart

Requires an NVIDIA GPU with Docker GPU support and just⁠.

git clone https://github.com/underlabs-ai/decisor.git
cd decisor
cp .env.example .env

Set SGLANG_IMAGE=underlabsai/decisor-sglang:0.5.20 in .env.

Generate a key and save it as SGLANG_API_KEY in .env:

python3 -c 'import secrets; print(secrets.token_urlsafe(32))'

Download the image and start:

just sglang-pull
just sglang-up
just sglang-status

Wait for the model to finish downloading and loading, then try:

just demo

See the GitHub repository⁠ for configuration, examples and image build instructions.

Technical preview. Keep the API key private and the engine port bound to localhost. Use HTTPS when accessing it over a network.