danielrosehill/voxtral-rocm repository overview

⁠Voxtral ROCm

Voxtral-Mini-3B-2507 speech-to-text transcription optimized for AMD GPUs with ROCm.

⁠Features

• Mistral Voxtral-Mini-3B model
• ROCm 6.4.4 + PyTorch 2.7.1
• 30-second audio chunking
• ~9.4GB VRAM
• 0.24x real-time (faster than real-time)

⁠Requirements

• AMD GPU with ROCm
• ~12GB VRAM
• Docker

⁠Usage

docker run --rm --device /dev/kfd --device /dev/dri -e HSA_OVERRIDE_GFX_VERSION=11.0.1 -v ./audio.mp3:/app/input.mp3 danielrosehill/voxtral-rocm python test/test_voxtral.py /app/input.mp3

⁠Source

https://github.com/danielrosehill/Local-Multimodal-Transcriber⁠