Sign inSign up

btbtyler09/mi100-training

By btbtyler09

•Updated 16 days ago

Image
0

120

btbtyler09/mi100-training repository overview

⁠mi100-training

Training containers for fine-tuning LLMs on AMD Instinct MI100 GPUs (gfx908, 32GB HBM2).

All images are built on btbtyler09/vllm-dev-mi100+AITER (ROCm 7.0 + PyTorch 2.9.1 + Triton 3.4.0 + CK flash_attn 2.8.3 + AITER).

⁠Available Tags

TagModelDescription
qwen3.5Qwen 3.5 (0.8B-4B)LoRA fine-tuning with Liger FLCE + fla + Triton flash attention

⁠Common Setup

All images require these docker flags for MI100 GPU access:

docker run -d \
    --device /dev/kfd --device /dev/dri \
    --group-add video --cap-add SYS_PTRACE \
    --shm-size=16g \
    ...

⁠qwen3.5

Solves the three OOM bottlenecks of Qwen 3.5 training on MI100:

BottleneckCauseSolutionSavings
248K vocab logitsseq × 248320 × 2 bytesLiger Kernel FLCE~8 GB
GatedDeltaNet (18 layers)PyTorch fallback upcasts to fp32flash-linear-attention (fla)Several GB
Full attention (6 layers)seq² × heads × 2 bytesTriton flash attention (AMD)~8 GB

Result: ~5-7 GB VRAM per GPU (down from 22+ GB OOM). Tested with Qwen3.5-0.8B at 16384 seq length on 4x MI100.

⁠Packages
PackageVersionPurpose
transformers>=5.0.0Model loading, training loop
peft>=0.15.0LoRA adapters
trl>=0.28.0SFTTrainer
liger-kernelgit mainFused Linear Cross Entropy for 248K vocab
flash-linear-attention>=0.4.0Triton kernels for GatedDeltaNet layers
deepspeed>=0.18.0ZeRO-2 optimizer/gradient sharding
accelerate>=1.0.0Multi-GPU launch
datasets>=3.0.0Dataset loading
bitsandbytes>=0.43.0Quantization support
⁠Required Environment Variables
FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE    # Use Triton flash attn (CK crashes on gfx908)
PYTORCH_ALLOC_CONF=garbage_collection_threshold:0.6  # Aggressive GC (expandable_segments unsupported on HIP 7.0)
ROCM_ARCH=gfx908
OMP_NUM_THREADS=16                        # Adjust for your CPU
⁠Quick Start
docker run -d --name training \
    --device /dev/kfd --device /dev/dri \
    --group-add video --cap-add SYS_PTRACE \
    --shm-size=16g \
    -v $(pwd):/workspace \
    -e FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE \
    -e PYTORCH_ALLOC_CONF=garbage_collection_threshold:0.6 \
    -e ROCM_ARCH=gfx908 \
    -e OMP_NUM_THREADS=16 \
    btbtyler09/mi100-training:qwen3.5 \
    accelerate launch --num_processes 4 /workspace/your_training_script.py
⁠Known Issues
  • causal-conv1d: Cannot pip install (requires nvcc). The GatedDeltaNet conv1d uses a pure PyTorch fallback with negligible memory impact. fla's chunk_gated_delta_rule kernel still >
  • Triton AMD varlen: varlen_fwd has a cu_seqlens shape mismatch with transformers' _upad_input. The image includes /workspace/mi100_patches.py to fix this. In your training s>
    import mi100_patches
    mi100_patches.patch_flash_attention()
    
  • expandable_segments: Not supported on HIP/ROCm 7.0. Use garbage_collection_threshold instead.
  • liger-kernel: PyPI release (0.7.0) lacks Qwen 3.5 support. Must install from git main

Tag summary

Content type

Image

Digest

sha256:a094edc17…

Size

6.4 GB

Last updated

16 days ago

docker pull btbtyler09/mi100-training:2026.06-validated