k8bit/higgs-audio repository overview

⁠Higgs Audio v3 TTS — Deployment Guide

⁠Overview

k8bit/higgs-audio is a Docker image providing an OpenAI-compatible REST API for Higgs Audio v3 TTS⁠ — a 4B parameter expressive TTS model with zero-shot voice cloning, 100+ language support, and inline emotion/prosody control tags.

The API is a drop-in replacement for OpenAI's /v1/audio/speech endpoint.

Key specs:

• Model: multimodalart/higgs-audio-v3-tts-4b-transformers (~9 GB, downloads on first start)
• Audio codec: bosonai/higgs-audio-v2-tokenizer (~800 MB, downloads on first start)
• Sample rate: 24 kHz
• Output formats: WAV, MP3, FLAC, OGG
• VRAM: ~11 GB (bf16) · ~6–7 GB (8-bit) · ~3.5 GB (4-bit)
• GPU: NVIDIA RTX 20xx or newer required

⁠Unraid Deployment

⁠Prerequisites

• Unraid 6.12+ with the NVIDIA Driver plugin installed
• Community Applications plugin
• An NVIDIA GPU with ≥6 GB VRAM

⁠Installing the container

⁠Option A — via Community Applications

Search for higgs-audio in the CA store. If the template is not yet listed, use Option B.

⁠Option B — manual XML template

1. In Unraid, go to Docker → Add Container → Template URL and paste:

https://raw.githubusercontent.com/k8bit/docker-templates/main/higgs-audio.xml

Or install manually: download higgs-audio-unraid.xml, open Unraid's Docker tab, click Add Container, then select Template Repository and import the file.

1. If importing the raw XML: go to Docker → Templates → Add New Template, paste the XML contents into the editor, save, then use it to create a container.

⁠Required path mappings

The /reference_voices path can be shared with Fish Speech — both services use the same NAME.wav + NAME.txt format.

⁠Key settings to review before first start

⁠First start

The container downloads two models on first launch:

• Main TTS model (~9 GB) — takes 5–15 min depending on connection speed
• Audio codec (~800 MB) — downloads on the first synthesis request

Watch progress:

docker logs -f higgs-audio-TTS

Ready when you see:

INFO  higgs-audio    Model ready.
INFO:     Uvicorn running on http://0.0.0.0:8000

⁠Docker Deployment (standalone)

⁠Prerequisites

• Docker Engine ≥ 24 with the Compose plugin
• NVIDIA Container Toolkit (nvidia-ctk, nvidia-container-runtime)
• CUDA 12.x drivers on the host

Verify GPU access before deploying:

docker run --rm --gpus all nvidia/cuda:12.8.1-base-ubuntu24.04 nvidia-smi

⁠Directory layout

/mnt/user/ai/
├── higgs-audio/
│   └── models/          # HuggingFace cache (~10 GB total)
└── voices/              # Reference audio pairs: NAME.wav + NAME.txt

⁠docker-compose.yml

name: higgs-audio

services:
  higgs-audio:
    image: k8bit/higgs-audio:latest
    container_name: higgs-audio
    restart: unless-stopped
    ports:
      - "${API_PORT:-8020}:8000"
    volumes:
      - ${MODELS_DIR:-/mnt/user/ai/higgs-audio/models}:/models
      - ${VOICES_DIR:-/mnt/user/ai/voices}:/reference_voices:ro
    environment:
      MODEL_REPO:           ${MODEL_REPO:-multimodalart/higgs-audio-v3-tts-4b-transformers}
      HIGGS_TTS_PRECISION:  ${PRECISION:-auto}
      HIGGS_NO_COMPILE:     ${NO_COMPILE:-1}
      HF_TOKEN:             ${HF_TOKEN:-}
      NVIDIA_VISIBLE_DEVICES: ${NVIDIA_VISIBLE_DEVICES:-all}
      NVIDIA_DRIVER_CAPABILITIES: compute,utility
      HF_HUB_DISABLE_XET:   1
    runtime: nvidia
    healthcheck:
      test: ["CMD", "curl", "-sf", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 10
      start_period: 300s

⁠.env

# Volumes
MODELS_DIR=/mnt/user/ai/higgs-audio/models
VOICES_DIR=/mnt/user/ai/voices

# Port
API_PORT=8020

# Hardware
NVIDIA_VISIBLE_DEVICES=all     # or a GPU UUID from: nvidia-smi -L

# Precision: auto | bf16 | 8bit | 4bit
PRECISION=auto

# Disable torch.compile — recommended; adds 60s warm-up if enabled
NO_COMPILE=1

# HuggingFace token (leave empty unless using private/gated repos)
HF_TOKEN=

⁠Starting

docker compose up -d
docker logs -f higgs-audio

⁠Voice library

Higgs Audio uses zero-shot voice cloning — no pre-encoding step is required. The model reads the reference clip directly on each request.

⁠Reference audio format

Each voice needs two files with matching stems:

/mnt/user/ai/voices/
├── scarlett.wav       # 5–30 seconds of clean speech
├── scarlett.txt       # exact transcript of the clip
├── walken.wav
└── walken.txt

• Supported formats: WAV, MP3, FLAC
• Optimal clip length: 10–30 seconds (clips over 30 s are trimmed automatically)
• Transcript must match the audio exactly — mismatches degrade cloning quality
• Filenames with spaces are supported

⁠Adding voices at runtime

# Upload a new reference clip
curl -X POST http://trogdor:8020/v1/audio/voices \
  -F "voice_id=mycustomvoice" \
  -F "transcript=Exact words spoken in the audio clip" \
  -F "audio=@/path/to/clip.wav"

The voice is immediately available for synthesis. No restart needed.

⁠Listing voices

curl http://trogdor:8020/v1/audio/voices | jq '.voices[].name'

⁠API Reference

⁠Health check

curl http://trogdor:8020/health

{
  "status": "ok",
  "model_loaded": true,
  "voices": 102,
  "hardware": "NVIDIA GeForce RTX 3090 (24.0 GB)"
}

⁠Synthesise speech

curl -X POST http://trogdor:8020/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "higgs-audio",
    "input": "Hello, this is a test of the Higgs Audio TTS system.",
    "voice": "scarlett",
    "response_format": "mp3",
    "speed": 1.0
  }' \
  --output test.mp3

⁠Extended parameters

Beyond the standard OpenAI fields (model, input, voice, response_format, speed), the API accepts:

⁠Emotion and prosody tags

Inline tags control expression. Place them in the input text:

{
  "input": "<|excited|>We did it! <|calm|>Now let's take a breath and review what happened."
}

Common tags: <|excited|>, <|sad|>, <|whispering|>, <|calm|>, <|angry|>, <|laughing|>, <|serious|>, <|fast|>, <|slow|>.

⁠Multi-speaker synthesis

curl -X POST http://trogdor:8020/v1/audio/speech/multi \
  -H "Content-Type: application/json" \
  -d '{
    "turns": [
      {"text": "Good morning! How are you today?", "voice": "scarlett"},
      {"text": "I'm doing well, thanks for asking.", "voice": "walken"}
    ]
  }' \
  --output dialogue.wav

⁠Environment variable reference

⁠Troubleshooting

⁠Model download fails — Xet error

Symptom: RuntimeError: Task error: File reconstruction error: CAS Client Error

Cause: HuggingFace's Xet storage protocol is failing in your network environment.

Fix: Set HF_HUB_DISABLE_XET=1 in the container environment (already the default in k8bit/higgs-audio:latest). To manually download the audio codec when already running:

docker exec -e HF_HUB_DISABLE_XET=1 higgs-audio-TTS python3 -c "
from huggingface_hub import snapshot_download
snapshot_download('bosonai/higgs-audio-v2-tokenizer', cache_dir='/models/hub', ignore_patterns=['*.png'])
print('Done')
"
docker restart higgs-audio-TTS

⁠Speech generation returns HTTP 500 — audio codec error

Symptom: Can't load the model for 'bosonai/higgs-audio-v2-tokenizer'

Cause A: The audio codec model has not been downloaded yet. It downloads on the first synthesis call, which can be slow on first run. Wait for the download to complete.

Cause B: The hf-xet package is installed and failing (see above).

Cause C: transformers version < 5.14 — the HiggsAudioV2TokenizerModel class is only registered natively in transformers ≥ 5.14. The current image sets transformers>=5.14. If running an older image, upgrade inside the container:

docker exec higgs-audio-TTS pip install --break-system-packages --upgrade transformers
docker restart higgs-audio-TTS

⁠CORS errors in browser — "Failed to fetch"

Symptom: All API calls from a locally-opened HTML file return "Failed to fetch".

Cause: The server was missing CORSMiddleware. This is fixed in k8bit/higgs-audio:latest.

If running an older image, apply the live patch:

docker exec higgs-audio-TTS python3 -c "
p = open('/app/server.py').read()
if 'CORSMiddleware' in p:
    print('Already patched')
else:
    p = p.replace('from fastapi import FastAPI,', 'from fastapi.middleware.cors import CORSMiddleware\nfrom fastapi import FastAPI,')
    p = p.replace('lifespan=lifespan,\n)\n', 'lifespan=lifespan,\n)\n\napp.add_middleware(CORSMiddleware, allow_origins=[\"*\"], allow_methods=[\"*\"], allow_headers=[\"*\"])\n')
    open('/app/server.py', 'w').write(p)
    print('Patched')
"
docker restart higgs-audio-TTS

⁠Container crashes immediately — python-multipart

Symptom: RuntimeError: Form data requires "python-multipart" to be installed

Cause: Old image predating the python-multipart fix. Update to the latest image.

⁠First synthesis is very slow

The audio codec (bosonai/higgs-audio-v2-tokenizer) is loaded lazily on the first call. Expect 30–120 seconds for the first request while the codec loads and warms up. Subsequent calls are significantly faster.

Flash Attention 2 is not yet supported for this model architecture. Standard SDPA attention is used — throughput is lower than it would be with flash attention enabled.

⁠torch.compile warm-up delay

If HIGGS_NO_COMPILE=0, torch.compile runs on first inference and takes ~60 seconds. This is normal. Set HIGGS_NO_COMPILE=1 (the default) to skip compilation — recommended for Unraid where containers restart frequently.

⁠Sharing the voice library with Fish Speech

Both Fish Speech and Higgs Audio use reference audio pairs (NAME.wav + NAME.txt). They can share the same host directory:

# Fish Speech
- /mnt/user/ai/voices:/reference_voices:ro

# Higgs Audio
- /mnt/user/ai/voices:/reference_voices:ro

Fish Speech pre-encodes .s2voice profiles from these files; Higgs Audio reads the raw WAV directly. There is no conflict — the .s2voice files are ignored by Higgs Audio.