lovewin10/sglang-v13 repository overview

⁠sglang-v13

Modified lmsysorg/sglang:nightly-dev-cu13-20260924-ffac53d7⁠ by lovewin10. Not an official SGLang image.

It adds support for the qwen_dense_v13 architecture (Qwen3_5V13ForConditionalGeneration), for example LTM-Lab/Qwen3.8-27B-V13-b1⁠. The launch command is the same as for a Qwen3.5 model; only the image name changes.

⁠What is changed

• sglang/srt/models/qwen3_5_v13.py: the V13 architecture name registered on SGLang's native Qwen3.5 dense VL model. The checkpoint's world_model.* tensors are skipped.
• The V13 name is added wherever SGLang matches Qwen3_5ForConditionalGeneration (15 places), including:
  • the EAGLE/MTP draft model mapping (Qwen3_5ForCausalLMMTP);
  • the hybrid Gated DeltaNet mamba radix-cache and extra-buffer lists;
  • the Qwen VL multimodal processor.
• Everything else is the base image, unchanged.

The episodic KV memory and the world model of qwen_dense_v13 are not included. SGLang serves the backbone, the vision tower and the MTP head; the memory runs only through the model's Transformers code.

⁠Run

docker run --rm -it --gpus all --ipc=host -p 8000:8000 `
  -v "<folder containing the model>:/model:ro" `
  lovewin10/sglang-v13:nightly-dev-cu13-20260924-ffac53d7-v13 `
  sglang serve --model-path /model/Qwen3.8-27B-V13-b1 --trust-remote-code `
  --host 0.0.0.0 --port 8000 --mem-fraction-static 0.78 --served-model-name EVE-27B `
  --chat-template /model/Qwen3.8-27B-V13-b1/chat_template.jinja `
  --reasoning-parser qwen3-thinking --tool-call-parser qwen3_coder `
  --attention-backend flashinfer --linear-attn-backend triton --mamba-ssm-dtype float32 `
  --speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 `
  --enable-linear-replayssm-spec --mamba-radix-cache-strategy extra_buffer_lazy `
  --context-length 262144 --max-prefill-tokens 32768 --max-running-requests 8 `
  --kv-cache-dtype fp8_e4m3 --max-mamba-cache-size 16

Tested on one 96 GB GPU with Qwen3.8-27B-V13-b1: about 52 tokens/s for a single request, with EAGLE acceptance length 2.1–2.7; reasoning and tool calls come back parsed.

⁠Notes

• The patch anchors match this nightly build only.
• Base image license and terms apply (SGLang: Apache-2.0; the NVIDIA container license shown at startup).