xiaoyizhang/mage-vl-cowork repository overview

⁠Mage-VL-Cowork inference

GPU inference runtime for the Mage-VL-Cowork computer-use model, in the Mage model family⁠. Model weights are not included in the container. Mount a compatible, reviewed Hugging Face model directory separately.

⁠Image

docker.io/xiaoyizhang/mage-vl-cowork:0.1.0

Platform: Linux x86-64 (linux/amd64). Requires an NVIDIA GPU, NVIDIA Container Toolkit, and a driver compatible with CUDA 12.8.

Runtime versions: Python 3.11.15, SGLang 0.5.2 with Mage-compatible model code, PyTorch 2.8.0+cu128, Transformers 5.7.0, FlashInfer 0.3.1. Exact dependencies and a source-file checksum manifest are included at /opt/mage-release/.

⁠Run

Set MODEL_DIR to an absolute path containing the complete compatible model export: weights, configuration, tokenizer, processor, and model implementation.

docker pull xiaoyizhang/mage-vl-cowork:0.1.0

docker run --rm --name mage-vl-cowork \
  --gpus 'device=0' --shm-size=8g \
  -p 127.0.0.1:30001:30001 \
  --mount "type=bind,src=${MODEL_DIR},dst=/models/Mage-VL-Cowork,readonly" \
  xiaoyizhang/mage-vl-cowork:0.1.0

The model is served as Mage-VL-Cowork. Use the native /generate endpoint with explicit vision placeholders and image_data for multimodal inference. The service also exposes /health, /v1/models, and /v1/chat/completions.

Important: with the checkpoint's default legacy chat-template selection, OpenAI-style image_url requests can return text without incorporating image tokens. Do not treat a successful chat HTTP response as verified image support. Use the native example below; single-image and three-image native requests were checked with this release. A complete agent should supply its own reviewed task prompt and action parser.

curl --fail http://127.0.0.1:30001/health
curl --fail http://127.0.0.1:30001/v1/models

Minimal native image request (one or more local PNG/JPEG files):

python3 - screenshot.png <<'PY'
import base64, json, pathlib, sys, urllib.request
images = [base64.b64encode(pathlib.Path(p).read_bytes()).decode() for p in sys.argv[1:]]
prompt = '<|im_start|>system\nYou are a computer-use assistant.<|im_end|>\n<|im_start|>user\n'
prompt += '<|vision_start|><|image_pad|><|vision_end|>\n' * len(images)
prompt += 'Describe the visible screen in the last image.<|im_end|>\n<|im_start|>assistant\n<think>\n'
payload = {'text': prompt, 'image_data': images,
           'sampling_params': {'temperature': 0, 'max_new_tokens': 512}, 'stream': False}
request = urllib.request.Request('http://127.0.0.1:30001/generate',
    data=json.dumps(payload).encode(), headers={'Content-Type': 'application/json'})
with urllib.request.urlopen(request, timeout=300) as response:
    print(json.load(response)['text'])
PY

The default profile uses one GPU, BF16, a 32,768-token context limit, FlashInfer attention, CUDA graphs, and torch.compile. Startup can take several minutes because kernels may need compilation. Model/processor-specific constraints can impose smaller practical limits.

Configuration environment variables include MODEL_PATH, SERVED_MODEL_NAME, PORT, DP_SIZE, MAX_RUNNING_REQUESTS, MEM_FRACTION_STATIC, ENABLE_CUDA_GRAPH, and ENABLE_TORCH_COMPILE. Additional SGLang arguments can be appended after the image name. Logs go to stdout/stderr. XDG/Hugging Face caches use /tmp/mage-cache; other frameworks may use their default runtime cache locations. Runtime-generated caches are not distributed as build artifacts.

⁠Safety and scope

• Keep the endpoint on localhost or a trusted private network. The default example does not configure authentication; use an authenticated gateway before exposing the API to other users.
• The launcher uses trust_remote_code; only mount model code you have reviewed and trust. Do not mount host credentials or a host Docker socket.
• This is an inference runtime, not a desktop execution sandbox or a complete benchmark harness. Computer-use actions need a separate, sandboxed executor and suitable approval controls.
• The image does not, by itself, reproduce a benchmark score. Agent prompts, action parsing, environment images, and evaluation code are separate inputs.
• This is a clean runtime rebuild validated by environment/source checks and GPU transport/generation smoke tests, not a bit-identical reproduction of a previous deployment. Greedy generated text differed in the comparison run; a full benchmark was not rerun. No benchmark-score equivalence is claimed.
• Original dependency licenses remain with their installed packages. The SGLang license is also provided at /opt/mage-release/SGLANG-LICENSE. NVIDIA base-image license terms apply.