k8bit/vibevoice-api-gpu repository overview

⁠VibeVoice API GPU Docker Image

A self-contained Docker image wrapping VibeVoice-API⁠ — a Python/PyTorch implementation of Microsoft's VibeVoice speech synthesis model. Exposes an OpenAI-compatible REST API and a dark HTML WebUI for zero-shot voice cloning TTS. Runs entirely on a single consumer GPU; no multi-GPU setup required.

Docker Hub: k8bit/vibevoice-api-gpu Upstream: github.com/vibevoice-community/VibeVoice-API⁠

⁠What's inside

The image bundles:

• vibevoice-api — the Python/PyTorch inference stack, running via uvicorn on an internal port. Implements an OpenAI-compatible /v1/audio/speech endpoint. Imports VibeVoiceForConditionalGenerationInference directly via the vibevoice-community/VibeVoice⁠ package.
• nginx — reverse proxy on port 8889. Serves the static WebUI at /, proxies /v1/ to uvicorn, and serves /voices/ for direct audio file access.
• FastAPI wrapper — a thin layer between nginx and the upstream synthesizer that applies a configurable post-processing pipeline (peak normalisation + silence prepend) to every generated WAV.
• WebUI — a dark HTML5 interface with tabs for TTS generation, voice library browsing, and API documentation.

Model weights are downloaded from HuggingFace at container first start and cached to a mounted volume. Nothing is baked into the image.

⁠Model variants

Both variants fit comfortably on a single 24 GB GPU (e.g. RTX 3090/4090). The 7B nf4 model offers noticeably better naturalness and prosody at essentially the same VRAM cost as the 1.5B in full precision. The 4-bit quantisation uses bitsandbytes nf4 and is loaded directly from pre-quantised HuggingFace weights — no on-the-fly quantisation.

Unlike most TTS systems, VibeVoice requires no reference transcript for voice cloning. It is a zero-shot cloner: point it at any clean WAV file and it will reproduce the speaker's voice. Just supply the audio.

⁠Requirements

• NVIDIA GPU (Turing or later recommended — sm_75+; Ampere sm_80+ for flash_attention_2)
• NVIDIA driver 550+
• NVIDIA Container Toolkit⁠ installed on the host
• Docker 20.10+
• ~10 GB free disk for the 1.5b variant; ~15 GB for 7b-4bit (model + Docker layers)

No CPU-only mode — this image requires a CUDA-capable GPU.

⁠Quick start

docker run -d \
  --name vibevoice-api \
  --runtime=nvidia \
  --gpus all \
  --shm-size=2g \
  -p 8889:8889 \
  -v /your/models:/models \
  -v /your/voices:/voices \
  k8bit/vibevoice-api-gpu:latest

On first start the 1.5B model (~3 GB) downloads to /models. Subsequent starts reuse the cache and load in 60–90 seconds while PyTorch initialises. Access the WebUI at http://localhost:8889/.

To use the 7B 4-bit model instead:

docker run -d \
  --name vibevoice-api-7b \
  --runtime=nvidia \
  --gpus all \
  --shm-size=2g \
  -p 8889:8889 \
  -v /your/models:/models \
  -v /your/voices:/voices \
  -e VIBEVOICE_MODEL_VARIANT=7b-4bit \
  k8bit/vibevoice-api-gpu:latest

⁠Environment variables

⁠Model selection

⁠Inference tuning

⁠Audio post-processing

Every response passes through a two-stage pipeline before being returned:

⁠Authentication (optional)

⁠HuggingFace

⁠Volumes

⁠API reference

⁠TTS — POST /v1/audio/speech

{
  "model":      "vibevoice-1.5b",
  "input":      "The quick brown fox.",
  "voice":      "kate",
  "ddpm_steps": 20,
  "cfg_scale":  1.5,
  "seed":       42
}

Returns audio/wav. Fields:

Voice cloning note: VibeVoice is a zero-shot cloner. Unlike ICL-based systems (e.g. Qwen3-TTS), no reference transcript is needed — the model infers speaker characteristics directly from the audio signal. A 5–30 second clean mono recording at 16–48 kHz works well.

⁠Other endpoints

GET  /health              → {"status":"ok","variant":"1.5b","model":"/models/..."}
GET  /v1/models           → OpenAI-compatible model list
GET  /v1/voices/aliases   → {"aliases":["kate","james",...],"count":2}

⁠cURL examples

⁠Basic TTS

curl http://localhost:8889/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"vibevoice-1.5b","input":"Hello from VibeVoice."}' \
  -o speech.wav

⁠Voice cloning — by alias

Place kate.wav in your /voices volume, then:

curl http://localhost:8889/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "vibevoice-1.5b",
    "input": "This is a cloned voice.",
    "voice": "kate",
    "seed":  42
  }' -o clone.wav

⁠Voice cloning — by path

curl http://localhost:8889/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{
    "model":      "vibevoice-1.5b",
    "input":      "Higher quality with more diffusion steps.",
    "voice_path": "/voices/kate.wav",
    "ddpm_steps": 30,
    "cfg_scale":  1.8
  }' -o clone_hq.wav

⁠Multi-speaker dialog

curl http://localhost:8889/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{
    "model":    "vibevoice-1.5b",
    "input":    "Speaker 1: Good morning.\nSpeaker 2: Morning! How are you?",
    "speakers": ["kate", "james"]
  }' -o dialog.wav

⁠Reproducible batch generation

Pin a seed to get identical output for the same prompt across multiple runs:

for i in 1 2 3; do
  curl http://localhost:8889/v1/audio/speech \
    -H 'Content-Type: application/json' \
    -d '{"model":"vibevoice-1.5b","input":"Take one.","voice":"kate","seed":100}' \
    -o "take_${i}.wav"
done
# All three files will be byte-identical.

⁠WebUI

The WebUI is served at http://<host>:8889/ and requires no login.

• TTS tab — text input, voice picker, sliders for DDPM steps / CFG scale / speed, seed field. Click a voice alias to clone it; the alias is sent in the API request automatically.
• Voices tab — browse all aliases registered from your /voices volume.
• API tab — inline documentation and cURL examples for every endpoint.

The header shows a status indicator (loading / ready) and a chip displaying the active variant (e.g. 7b-4bit) once the model finishes loading.

⁠Docker Compose example

services:
  vibevoice-api:
    image: k8bit/vibevoice-api-gpu:latest
    runtime: nvidia
    environment:
      NVIDIA_VISIBLE_DEVICES: all
      NVIDIA_DRIVER_CAPABILITIES: compute,utility
      VIBEVOICE_MODEL_VARIANT: "1.5b"   # or "7b-4bit"
      VIBEVOICE_DDPM_STEPS: "10"
      VIBEVOICE_SEED: "42"
      NORM_TARGET_DB: "-1.0"
      SILENCE_SEC: "0.5"
    ports:
      - "8889:8889"
    volumes:
      - ./models:/models
      - ./voices:/voices
      - ./hf-cache:/cache/huggingface
    shm_size: "2gb"
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-sf", "http://localhost:8889/health"]
      interval: 30s
      timeout: 10s
      start_period: 300s
      retries: 10

⁠Voice library conventions

Any WAV file in your /voices mount is automatically registered as a voice alias on container start. The filename stem becomes the alias:

voices/
  kate.wav          # alias: "kate"
  announcer.wav     # alias: "announcer"
  deep_male.wav     # alias: "deep_male"

For best cloning results:

• 5–30 seconds of clean speech (longer is generally better)
• Mono or stereo (mono preferred)
• 16–48 kHz sample rate
• Minimal background noise or music
• A single consistent speaker throughout the clip

No companion .txt transcript file is needed. VibeVoice infers voice characteristics directly from the audio.

⁠Tuning DDPM steps

The diffusion decoder runs a fixed number of denoising steps per generation. The default of 10 balances speed and quality well for most use cases. If output sounds slightly rough or unnatural, try increasing steps. Diminishing returns typically set in around 30–40.

These multipliers are approximate and vary with text length.

⁠Performance notes

• VRAM: 1.5b bfloat16 uses 6–8 GB; 7b-4bit uses ~6.6 GB. Both leave ample headroom on a 24 GB card.
• Flash Attention 2: Used automatically on Ampere (sm_80+) and later GPUs if the flash-attn package is installed. Falls back to SDPA transparently. The image does not pre-install flash-attn due to its long compile time; you can extend the image to add it.
• Concurrency: Each request holds the GPU for the full diffusion pass. VIBEVOICE_MAX_CONCURRENCY=2 allows requests to queue without the server rejecting them. On a single-GPU setup, they execute serially; there is no real parallelism for GPU-bound work.
• Cold start: The first synthesis after container start may be slower as PyTorch JIT-compiles kernels. Subsequent requests are faster.

⁠Building from source

The build script generates a multi-stage-free Dockerfile targeting nvidia/cuda:12.8.0-runtime-ubuntu22.04:

1. Installs PyTorch cu121 first (to prevent pip from substituting a CPU build when resolving the vibevoice-api dependency tree)
2. Installs vibevoice (community model code) and vibevoice-api (server package) from their respective GitHub repos
3. Installs bitsandbytes for 4-bit quantised model loading
4. Writes our FastAPI wrapper (main.py), nginx config, entrypoint, and static WebUI

CUDA architectures compiled into the PyTorch wheel: sm_75, sm_80, sm_86, sm_89 (Turing, Ampere, Ada Lovelace).

bash build_vibevoice_api_gpu.sh

⁠Licence

This image packages VibeVoice-API⁠ and the vibevoice-community/VibeVoice⁠ model code. Refer to the upstream repositories for licence terms. The VibeVoice-1.5B and VibeVoice-7B model weights are subject to their respective HuggingFace repository licences. The 4-bit quantised weights at DevParker/VibeVoice7b-low-vram are released under the MIT licence.