OmniVoice.cpp is a C++/GGML port of the OmniVoice text-to-speech model by ServeurpersoCom. It supports zero-shot TTS across 646 languages, voice cloning from a reference audio clip, and voice design via natural language style instructions. Models are distributed as GGUF-quantised files, making the deployment lean and self-contained without requiring Python ML frameworks.
This deployment wraps the C++ binaries in a FastAPI layer that adds an OpenAI-compatible REST API, a voice library, and a browser-based testing UI. Two variants are provided: a GPU-accelerated deployment for production use and a CPU-only deployment for lower-resource or secondary servers.
The deployment uses two C++ binaries from omnivoice.cpp:
tts-server is a long-running HTTP server that loads the GGUF model once at startup and keeps it resident in memory (or VRAM on GPU). It exposes the core synthesis endpoint and handles repeated requests efficiently without reloading the model. This is the hot path for all live speech synthesis.
omnivoice-tts is a CLI tool used for voice cloning. It is invoked once per clone request, loading the model from disk each time. This makes it slower than tts-server but allows passing a reference audio clip and transcript to clone a speaker's voice.
The FastAPI application sits in front of both binaries. It starts tts-server as a managed subprocess on startup, waits for it to become healthy, then accepts incoming connections. Synthesis requests are proxied directly to tts-server. Clone requests shell out to omnivoice-tts with the provided reference audio. A voice library allows saving named reference voices to disk so they can be reused without re-uploading.
Models are downloaded from CreativeHub008/OmniVoice-GGUF on HuggingFace. No token is required — the repository is public. Two files are always required per variant: a base language model and a tokenizer/codec model.
Available quantisation variants:
| Variant | Combined size | Notes |
|---|---|---|
| Q8_0 | ~900 MB | Near-lossless quality. Recommended for GPU. |
| Q4_K_M | ~500 MB | Very good quality at roughly half the size. Recommended for CPU. |
| BF16 | ~1.2 GB | Lossless. Maximum fidelity, higher VRAM requirement. |
| F32 | ~2.4 GB | Full precision. Reference/testing only. |
Models are downloaded to a dedicated volume on first container start. They are stored flat (not in a HuggingFace cache hierarchy), so the model volume should be separate from any shared HuggingFace cache used by other containers.
The GPU image is built on the NVIDIA CUDA devel base for compilation and the CUDA runtime base for the final image. The GGML CUDA backend is compiled in, providing GPU-accelerated inference for both the language model and the DAC codec.
The GGML backend for tts-server is set via the TTS_BACKEND environment variable (default CUDA0). Voice cloning via omnivoice-tts uses the CLONE_BACKEND variable (default CUDA0 — with 24 GB of VRAM the model fits comfortably alongside tts-server's resident copy). Either can be set to CPU if needed.
Typical performance: synthesis is near-realtime on a modern GPU. A 10-second audio segment generates in roughly 1–3 seconds on an RTX 3090.
Requirements: NVIDIA GPU with CUDA 12.x support, NVIDIA Container Toolkit installed on the host (on unRAID, this is provided by the Nvidia Driver plugin).
The CPU image is built on plain Ubuntu 22.04 with OpenBLAS enabled (GGML_BLAS=ON) for accelerated matrix operations. No GPU or CUDA libraries are required. The default model variant is Q4_K_M to balance quality against RAM usage and inference speed.
The TTS_BACKEND and CLONE_BACKEND variables both default to CPU. Synthesis is slower than GPU — expect roughly 5–20× real-time depending on core count and CPU generation. For a radio station context the CPU deployment is best suited to pre-rendering segments rather than live synthesis.
Requirements: No GPU required. Recommended minimum 4 CPU cores and 4 GB free RAM for Q4_K_M. Allow 8 GB RAM for Q8_0.
Both deployments expose the same API surface. The service runs on port 8880 by default (the CPU variant is conventionally mapped to port 8881 on the host to allow both to run on the same server).
| Endpoint | Method | Description |
|---|---|---|
/v1/audio/speech | POST | OpenAI-compatible TTS. Accepts input, response_format (wav or pcm), and instructions for voice style. Proxied to tts-server (fast path). |
/v1/audio/clone | POST | Voice cloning. Accepts text, a reference WAV upload or saved voice name, and a reference transcript. Runs omnivoice-tts (slower). |
/v1/voices | GET | List saved voices in the voice library. |
/v1/voices/{name} | POST | Save a reference WAV and transcript to the voice library. |
/v1/models | GET | OpenAI-compatible model list. |
/health | GET | Health check. |
/web | GET | Browser-based testing UI. |
/docs | GET | Swagger / OpenAPI documentation. |
The /v1/audio/speech endpoint is a drop-in replacement for the OpenAI TTS API and is compatible with clients such as Open WebUI, SillyTavern, and similar tools.
Named reference voices can be saved to a shared volume. Each entry is a WAV file paired with a .txt transcript of the same name. Voices saved here are available by name in the clone endpoint and appear in the WebUI Voice Library tab, removing the need to re-upload reference audio on each clone request.
The voices volume can be shared across multiple containers (for example, with other TTS services on the same server) so a single library of reference speakers is maintained in one place.
Both deployments include a dark-themed browser UI reachable at /web. It has three tabs:
Speech — Enter text and an optional voice style instruction (e.g. male, deep, professional broadcaster). Generates audio via the fast tts-server path. Suitable for live testing of synthesis quality and style control.
Voice Clone — Upload a reference audio clip (3–10 seconds of clean single-speaker audio) with its transcript, or select a saved voice from the library. Generates audio in the cloned voice. On the GPU deployment this runs on GPU; on CPU expect minutes for longer segments.
Voice Library — Upload and name reference voices. These are saved to the shared voices volume and immediately available for clone requests via the API or the Clone tab.
| Container path | Purpose |
|---|---|
/models | GGUF model files. Downloaded on first start. Dedicated per deployment — do not share with a general HuggingFace cache. |
/voices | Voice library (WAV + transcript pairs). Can be shared across TTS containers on the same server. |
| Variable | GPU default | CPU default | Description |
|---|---|---|---|
HF_MODEL_ID | CreativeHub008/OmniVoice-GGUF | same | HuggingFace repo to download models from. |
MODEL_VARIANT | Q8_0 | Q4_K_M | GGUF quantisation variant. |
LANGUAGE | English | English | Global language for tts-server. |
TTS_BACKEND | CUDA0 | CPU | GGML backend for the tts-server process. |
CLONE_BACKEND | CUDA0 | CPU | GGML backend for omnivoice-tts clone requests. |
PORT | 8880 | 8880 | Internal port the FastAPI app listens on. |
Both variants are available as Community Applications templates. Install via the Docker tab in the unRAID UI. On first container start, models are automatically downloaded from HuggingFace and saved to the configured models path. Subsequent starts skip the download if both model files are already present.
The GPU template requires the Nvidia Driver plugin and the NVIDIA Container Toolkit to be active on the host. The CPU template has no special host requirements and can be installed on any unRAID server.
Both containers expose their web UI and API on the configured host port. Access the testing UI at http://[server-ip]:[port]/web and the API documentation at http://[server-ip]:[port]/docs.
Content type
Image
Digest
sha256:7280e73a4…
Size
86.6 MB
Last updated
4 months ago
docker pull k8bit/omnivoice-cpp-cpu