abtools/omnivoice-cuda repository overview

OmniVoice text-to-speech for Home Assistant on NVIDIA Jetson (arm64, JetPack 7, CUDA 13).

Wraps rhasspy/wyoming-piper with its OmniVoice backend, but runs the WHOLE OmniVoice pipeline on the GPU in bfloat16 (language model, audio codec and sampling loop) instead of CPU/ONNX. Upstream ships OmniVoice for amd64 only; on a Jetson AGX Orin this image goes from ~9 s to under 1 s until the first audio.

• Base: nvidia/cuda:13.0 runtime (Ubuntu 24.04), PyTorch 2.10 cu130 aarch64 wheels
• Wyoming protocol on port 10200; add it to Home Assistant as a Wyoming integration
• Mount /data for the model cache (~3.6 GB, downloaded on first start) and custom voices under /data/voices/// (instruct.txt for voice design, or ref.wav + ref.txt for cloning)
• GPU is enabled automatically (--use-cuda); pass backend options as the container command, e.g. --backend omnivoice --omnivoice-language German --omnivoice-steps 10

docker run -d --runtime nvidia -p 10200:10200 -v /data/omnivoice:/data
abtools/omnivoice-cuda:1.0.0 --backend omnivoice --omnivoice-language German --omnivoice-steps 10

Dockerfile and the GPU patch are documented in the image's build directory; the patch asserts on the upstream code it modifies so a version bump fails loudly rather than silently falling back to CPU.

Provided by AB-SmartHouse⁠.