farbodatdocker/scalarlm repository overview

⁠ScalarLM for OTel (AMD MI355X, NVIDIA H100, AWS Trainium2)

One container that serves an OpenAI compatible vLLM endpoint and runs Megatron training through Slurm. Supports SFT, LoRA, full parameter training and GRPO with ddp, fsdp and pytorch_fsdp.

⁠Tags

There is no latest tag. Always pull a hardware tag.

⁠Quick start (NVIDIA H100)

docker run -d --name scalarlm --gpus all --ipc host --shm-size=64g \
  -e SCALARLM_MODEL=Qwen/Qwen3-0.6B -p 8000:8000 -p 8001:8001 \
  -v $HOME/.cache/huggingface:/root/.cache/huggingface \
  farbodatdocker/scalarlm:h100-v2.1
curl http://localhost:8000/v1/health

⁠Quick start (AMD MI355X)

docker run -d --name scalarlm --init --shm-size=64g --ulimit memlock=-1 \
  --device=/dev/kfd --device=/dev/dri --group-add video \
  --security-opt seccomp=unconfined -p 8000:8000 -p 8001:8001 \
  -e SCALARLM_MODEL=Qwen/Qwen3-0.6B -e SCALARLM_MAX_GPUS_PER_NODE=8 \
  farbodatdocker/scalarlm:mi355-v1.8

--shm-size=64g is required. SCALARLM_MAX_GPUS_PER_NODE defaults to 1.

⁠Quick start (AWS Trainium2)

Runs on 8 Trainium2 devices: vLLM uses the first 4 and training uses the rest.

docker run -d --name scalarlm \
  --device=/dev/neuron0 --device=/dev/neuron1 --device=/dev/neuron2 --device=/dev/neuron3 \
  --device=/dev/neuron4 --device=/dev/neuron5 --device=/dev/neuron6 --device=/dev/neuron7 \
  --shm-size=64g --ulimit memlock=-1 -p 8000:8000 -p 8001:8001 \
  -e SCALARLM_MODEL=/root/models/<serving-model> \
  -e SCALARLM_TENSOR_PARALLEL_SIZE=16 -e SCALARLM_MAX_GPUS_PER_NODE=16 \
  -e SCALARLM_VLLM_ARGS="<vLLM Neuron flags matching the compiled graphs>" \
  -v /path/to/models:/root/models:ro -v /path/to/hf-cache:/root/.cache/huggingface \
  farbodatdocker/scalarlm:trainium2-v1.0
curl http://localhost:8000/v1/health

Tested with Gemma 4 31B LoRA. The first start compiles for several minutes. Adapters cannot be loaded at run time, so merge them and restart with the merged model. See the Trainium2 guide for the vLLM flags.

⁠Documentation

• MI355X runbook⁠
• H100 runbook⁠
• Trainium2 guide⁠

⁠About OTel

Open Telco (OTel) AI is an open ecosystem of telecom datasets, benchmarks, models, and reproducible training and inference recipes. OTel’s mission is to give telecom operators the freedom to choose their models, software, hardware, and infrastructure. With contributions from across the industry, OTel has built a shared, open-source AI foundation that enables operators to build, adapt, and deploy AI on their own terms.

OTel has expanded beyond open telecom datasets and specialized embedding, reranking, language, decision, classification, and safety models to include a reproducible training and inference ecosystem. The ecosystem supports multiple software stacks across hardware from AMD, NVIDIA, AWS, Apple, Tenstorrent, and Intel.

OTel research includes a NeurIPS 2026 Spotlight paper⁠ in the Evaluations & Datasets Track and a Breakthrough Impact paper accepted to the ACM AI Leadership Summit 2026⁠. See selected blog and media coverage⁠ of OTel.

GitHub⁠ · Hugging Face⁠ · Paper⁠

Maintainer: Farbod Tavakkoli. License: Apache 2.0.