farbodatdocker/otel-decision-vllm-rocm repository overview

⁠OTel 2.0 Decision on AMD MI350P (vLLM ROCm)

The official vllm/vllm-openai-rocm:v0.30.0 image with a corrected Gemma 4 MoE kernel and an MI350P tile configuration. Built to serve OTel-2.0-Decision-26B-A4B-4bit⁠, a 4 bit telecom decision model for classification, Yes/No and ordered level questions.

The tag matches the upstream vLLM version.

⁠Run

hf download farbodtavakkoli/OTel-2.0-Decision-26B-A4B-4bit --local-dir ./model
docker run --rm --device /dev/kfd --device /dev/dri --group-add video \
  --ipc=host --security-opt seccomp=unconfined -p 8000:8000 \
  -v "$PWD/model:/model:ro" \
  farbodatdocker/otel-decision-vllm-rocm:v0.30.0 /model \
  --served-model-name OTel-2.0-Decision-26B-A4B-4bit \
  --max-model-len 16384 --enable-prefix-caching --max-logprobs 64 \
  --moe-backend flydsl --port 8000

The first start takes 4 to 10 minutes. The server has no authentication.

Prompting, label probabilities and benchmarks are on the model card⁠.

⁠About OTel

Open Telco (OTel) AI is an open ecosystem of telecom datasets, benchmarks, models, and reproducible training and inference recipes. OTel’s mission is to give telecom operators the freedom to choose their models, software, hardware, and infrastructure. With contributions from across the industry, OTel has built a shared, open-source AI foundation that enables operators to build, adapt, and deploy AI on their own terms.

OTel has expanded beyond open telecom datasets and specialized embedding, reranking, language, decision, classification, and safety models to include a reproducible training and inference ecosystem. The ecosystem supports multiple software stacks across hardware from AMD, NVIDIA, AWS, Apple, Tenstorrent, and Intel.

OTel research includes a NeurIPS 2026 Spotlight paper⁠ in the Evaluations & Datasets Track and a Breakthrough Impact paper accepted to the ACM AI Leadership Summit 2026⁠. See selected blog and media coverage⁠ of OTel.

GitHub⁠ · Hugging Face⁠ · Paper⁠

Maintainer: Farbod Tavakkoli. License: Apache 2.0.