tomorsogody/vibevoice-api repository overview

⁠Using tomorsogody/vibevoice-api

A ready-to-run, OpenAI-compatible text-to-speech server powered by VibeVoice⁠ (long-form, multi-speaker TTS). The image is public on Docker Hub — no login needed:

docker pull tomorsogody/vibevoice-api:latest

Point any OpenAI TTS client at it by changing the base_url.

⁠1. What you need

• An NVIDIA GPU with ≥12 GB VRAM for the 1.5B model (≥24 GB for 7B). The GPU must be Ampere or newer (RTX 30xx/40xx, A-series) — the model runs in bfloat16, which older Pascal/Turing cards (Titan Xp, RTX 20xx) don't support.
• Docker with the NVIDIA Container Toolkit (--gpus all support), or a rented GPU on Vast.ai / RunPod / Lambda / any GPU host.
• No model weights to download manually — they're pulled from HuggingFace on the first request into a cache volume (~5–6 GB for 1.5B, one time).

⁠2. Run it (own GPU box)

docker run --gpus all -p 8000:8000 \
  -e VV_API_KEY=choose-a-secret \
  -e VV_DEFAULT_MODEL=vibevoice-1.5b \
  -v vv-cache:/cache/huggingface \
  tomorsogody/vibevoice-api:latest

• VV_API_KEY — clients must send Authorization: Bearer <this>. Leave it unset and auth is OFF (open to anyone who can reach the port).
• -v vv-cache:/cache/huggingface — persists downloaded weights across restarts.
• First /v1/audio/speech call is slow (weight download); everything after is fast.

Check it's up:

curl http://localhost:8000/health          # {"status":"ok","engine":"vibevoice"}

⁠3. Run it on Vast.ai

• Image: tomorsogody/vibevoice-api:latest
• GPU: ≥12 GB, Ampere+ (a cheap RTX 3060 works) · Disk: ≥40 GB · Port: 8000
• Docker options: -p 8000:8000 -e VV_API_KEY=secret -e VV_DEFAULT_MODEL=vibevoice-1.5b -e HF_HOME=/cache/huggingface
• On-start (if launched as an SSH instance): cd /app && nohup uvicorn app.main:app --host 0.0.0.0 --port 8000 > /var/log/vibevoice.log 2>&1 &
• Your endpoint is the instance's mapped public IP:PORT for container port 8000.

Full walkthrough: docs/VASTAI.md⁠.

⁠4. Call the API

⁠OpenAI Python SDK

from openai import OpenAI
client = OpenAI(base_url="http://YOUR_HOST:8000/v1", api_key="choose-a-secret")

client.audio.speech.create(
    model="vibevoice-1.5b",     # or "vibevoice-7b"
    voice="alice",              # see /v1/voices
    input="Hello from VibeVoice!",
).stream_to_file("hello.mp3")

⁠curl

curl -X POST http://YOUR_HOST:8000/v1/audio/speech \
  -H "Authorization: Bearer choose-a-secret" \
  -H "Content-Type: application/json" \
  -d '{"model":"vibevoice-1.5b","voice":"alice","input":"Hello there.","response_format":"mp3"}' \
  --output hello.mp3

⁠Node (OpenAI SDK)

import OpenAI from "openai";
import fs from "fs";
const client = new OpenAI({ baseURL: "http://YOUR_HOST:8000/v1", apiKey: "choose-a-secret" });
const res = await client.audio.speech.create({ model: "vibevoice-1.5b", voice: "alice", input: "Hi!" });
fs.writeFileSync("hi.mp3", Buffer.from(await res.arrayBuffer()));

⁠5. API reference

/v1/audio/speech body:

⁠Multi-speaker (podcasts/dialogue)

{
  "model": "vibevoice-1.5b",
  "voice": "frank,maya",
  "input": "Speaker 1: Welcome to the show!\nSpeaker 2: Great to be here.",
  "response_format": "mp3"
}

Speaker 1 → first voice, Speaker 2 → second, up to 4 speakers.

⁠Expressiveness (no SSML tags)

VibeVoice has no [laugh]/[sigh] tags. Write cues phonetically in the text — "Ha ha ha!", "Ugh…", "Whoa!" — and its emergent expressiveness handles laughter/emotion naturally. Bracketed tags may be read literally.

⁠6. Performance & sizing

Measured on a single RTX 3060 12 GB (1.5B model):

• Real-time factor ~0.72–0.76 — generates audio faster than real-time.
• Rendered a 9m21s narration in 7m05s with no out-of-memory.
• Rule of thumb: output length ≈ words ÷ 4 seconds; compute ≈ 0.75 × audio length.
• For long-form (podcasts/audiobooks, 20 min+) use a 24 GB GPU for headroom.
• One generation runs at a time per GPU (requests queue); long renders can take minutes, so use generous client timeouts.

⁠7. Security (read before exposing publicly)

• VV_API_KEY is a single static bearer token — the only built-in auth. Unset = no auth. Rotate it if it leaks.
• The container serves plain HTTP. For anything public, put it behind a TLS reverse proxy (Caddy, nginx, or a Cloudflare Tunnel) and add rate limiting — a leaked key otherwise means unlimited GPU spend.
• Don't expose the raw port to the internet without the above.

⁠8. Configuration (all env vars are VV_-prefixed)

VV_ENGINE (vibevoice/mock), VV_API_KEY, VV_DEFAULT_MODEL, VV_MAX_LOADED_MODELS, VV_ATTN_IMPL (auto/sdpa/flash_attention_2), VV_CFG_SCALE, VV_MAX_CONCURRENCY, VV_REQUEST_TIMEOUT_S, VV_DEFAULT_VOICE. Full list: .env.example⁠.

Source & Dockerfile: https://github.com/ttomor/vibevoice-api⁠ · Model & license: VibeVoice⁠ (MIT).