k8bit/fish-speech repository overview

⁠Fish Speech S2 Pro — Self-Hosted Deployment Guide

Fish Speech S2 Pro is a state-of-the-art multilingual text-to-speech system capable of zero-shot voice cloning from a short reference audio clip. This container packages the model with the s2.cpp⁠ GGUF inference engine — a pure C++/GGML runtime with no Python or PyTorch dependency at inference time — alongside an OpenAI-compatible REST API and a Gradio web interface.

⁠Overview

The container runs three processes:

• s2.cpp server — pure C++ inference engine, internal port 3030 (never exposed)
• FastAPI — OpenAI-compatible API, port 8000
• Gradio — web UI for script input, voice library, and previewing, port 8001

On first start the container downloads the GGUF model from HuggingFace (~4.5 GB for the default q6_k quantization) and any missing voice profiles are pre-encoded from your reference audio library.

⁠Prerequisites

• Docker 24+ with the NVIDIA Container Toolkit installed (install guide⁠)
• NVIDIA GPU with CUDA 12.x support and at least 8 GB VRAM for q6_k (see VRAM table below)
• CPU-only operation is supported but significantly slower (set CUDA_DEVICE=-1)
• ~10 GB free disk for model + overhead

VRAM requirements at runtime (model weights + KV cache + compute buffers):

⁠Quick Start

docker run -d \
  --name fish-speech \
  --gpus all \
  -p 8007:8000 \
  -p 8008:8001 \
  -v /path/to/models:/models \
  -v /path/to/voices:/voices \
  -v /path/to/reference_voices:/reference_voices:ro \
  k8bit/fish-speech:latest

With Docker Compose:

cp .env.example .env
# Edit .env — set FISH_MODELS_DIR, FISH_VOICES_DIR, FISH_REF_VOICES_DIR
docker compose up -d

The model downloads automatically on first start. Allow 60–120 seconds for the model to load before the API becomes responsive. Check status:

curl http://localhost:8007/health

Expected response when ready:

{
  "status": "ok",
  "s2_server": "ok",
  "voices_encoded": 3,
  "voices_raw": 0,
  "sample_rate": 44100,
  "target_dbfs": -1.0
}

⁠Configuration Reference

⁠Volume Reference

⁠API Reference

⁠Health check

curl http://localhost:8007/health

⁠List available voices

curl http://localhost:8007/v1/audio/voices

⁠Basic speech synthesis

curl -X POST http://localhost:8007/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fish-speech",
    "input": "Welcome to the station. Up next, your morning headlines.",
    "voice": "presenter",
    "response_format": "wav"
  }' \
  --output output.wav

⁠Speech with emotion tags

Fish Speech S2 Pro supports 15,000+ inline emotion tags using [tag] syntax. Tags can be free-form natural language descriptions.

curl -X POST http://localhost:8007/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Breaking news. [pause] A major [excited] discovery has been made [pause] that could change everything.",
    "voice": "presenter",
    "response_format": "wav"
  }' \
  --output breaking_news.wav

Common tags: [pause] [whisper] [excited] [laughing] [sigh] [sad] [angry] [emphasis] [chuckle] [inhale] [volume up] [volume down] [professional broadcast tone] [shouting]

⁠Radio broadcast mode (sentence segmentation)

For long radio scripts, sentence segmentation produces more natural output by synthesising each sentence independently and inserting configurable pauses between them.

curl -X POST http://localhost:8007/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Good morning and welcome to the show. Today we cover the top stories of the day. Stay tuned for more after the break.",
    "voice": "presenter",
    "response_format": "wav",
    "segment_sentences": true,
    "sentence_pause_ms": 250
  }' \
  --output morning_show.wav

⁠On-the-fly voice cloning

Clone a new voice without pre-registering it by supplying reference audio as base64:

REF_B64=$(base64 -w0 reference.wav)

curl -X POST http://localhost:8007/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d "{
    \"input\": \"Text to synthesize in the cloned voice.\",
    \"reference_audio_b64\": \"${REF_B64}\",
    \"reference_text\": \"Exact words spoken in the reference audio.\",
    \"response_format\": \"wav\"
  }" \
  --output cloned.wav

Reference audio should be 5–30 seconds of clean, single-speaker audio.

⁠Upload a new voice to the library

curl -X POST http://localhost:8007/v1/audio/voices \
  -F "voice_id=alice" \
  -F "transcript=Hello, this is a sample of my voice for the voice cloning library." \
  -F "audio=@/path/to/alice_sample.wav" \
  -F "encode_now=false"

Set encode_now=true to pre-encode the .s2voice profile immediately (blocks while encoding, ~30 s). Otherwise the reference audio is forwarded with each request (slightly slower per request but available immediately).

⁠Voice Library Setup

The voice library allows you to pre-register reference audio clips that can be selected by name in any synthesis request. This is the recommended approach for radio stations with a fixed presenter roster.

Create a directory structure:

/mnt/user/ai/reference_voices/
├── presenter-alice.wav     # 5–30 s clean audio, mono or stereo
├── presenter-alice.txt     # exact transcript of what Alice says in the wav
├── reporter-bob.wav
├── reporter-bob.txt
└── newsreader.wav
    newsreader.txt

Mount this directory as /reference_voices:ro. On the first startup the container runs the s2.cpp CLI to encode each WAV+TXT pair into a .s2voice profile saved to /voices. On subsequent startups, existing profiles are reused (fast startup). The pre-encoding step takes roughly 30 seconds per voice.

Voice names derive from the filename stem. presenter-alice.wav becomes voice_id=presenter-alice.

⁠CPU vs GPU

With CUDA_DEVICE=0 (default) the full model is loaded to GPU. With CUDA_DEVICE=-1 or when no GPU is available, inference runs on CPU.

CPU performance on a modern server is roughly 10–20× slower than GPU. A 10-word sentence takes approximately 2–5 seconds on CPU versus 0.2–0.5 seconds on an RTX 3090 with q6_k. For radio automation where audio is pre-generated before broadcast, CPU operation is viable. For real-time interactive use, a GPU is strongly recommended.

To reduce VRAM usage while keeping GPU acceleration, set GPU_LAYERS to a number less than 36 (the total transformer layers). For example, GPU_LAYERS=18 offloads half the layers to GPU and runs the other half on CPU. The KV cache (~2.5 GB) is always allocated on GPU when any layers are offloaded.

⁠Model Management

The GGUF file is downloaded automatically on first start from rodrigomt/s2-pro-gguf on HuggingFace. To pre-supply the model (faster first start, useful for air-gapped environments):

# Download manually
huggingface-cli download rodrigomt/s2-pro-gguf \
  s2-pro-q8_0.gguf tokenizer.json \
  --local-dir /mnt/user/ai/fish-speech/models

Then set MODEL_FILE=s2-pro-q8_0.gguf in your environment.

To switch quantization, change MODEL_FILE and restart. The new file is downloaded automatically if not already present.

⁠Building from Source

# On Trogdor or any Docker host with GPU:
scp build_fish_speech.sh [email protected]:/mnt/user/Apps/temp/
nohup bash /mnt/user/Apps/temp/build_fish_speech.sh \
  > /mnt/user/Apps/temp/build_fish_speech.log 2>&1 &

# Monitor:
tail -f /mnt/user/Apps/temp/build_fish_speech.log

The build compiles s2.cpp from source with CUDA support in a devel image, then produces a lean runtime image. Build time is 10–20 minutes depending on the host.

⁠Troubleshooting

Container starts but /health returns connection refused

The s2.cpp server is still loading the model. Allow up to 120 seconds. Check internal logs with docker logs fish-speech.

s2-server: error: Cannot reach s2-server

The s2.cpp process crashed. Run docker exec fish-speech cat /var/log/s2-server.log to see the error. Common cause: insufficient VRAM for the selected quantization. Try a smaller GGUF (MODEL_FILE=s2-pro-q4_k_m.gguf) or reduce GPU_LAYERS.

Voice cloning sounds like the wrong speaker

Reference audio quality is the primary factor. Use a clean, noise-free 15–30 second clip with consistent speech. Avoid clips with background music, multiple speakers, or strong room reverb. The transcript must be exact — even small mismatches degrade clone quality.

First synthesis takes much longer than subsequent ones

Normal behaviour. The first request after startup exercises the full KV cache and warms up CUDA kernels. Subsequent requests are faster.

503 Service Unavailable during synthesis

s2.cpp handles one request at a time. The FastAPI server serialises requests using an asyncio lock. If you see 503 errors it means a previous synthesis is still in progress. Increase API_TIMEOUT if generating very long audio.

Audio is too quiet or clips

Adjust TARGET_DBFS. The default -1.0 targets just below full scale with a small headroom buffer. For radio broadcast where further processing (compression, limiting) is applied downstream, -3.0 or -6.0 leaves more headroom. The normalisation pipeline uses peak scaling with np.clip to guarantee no clipping regardless of the target.

Voice profiles not found after restart

Ensure the /voices volume is persisted between container restarts (not recreated). If using Docker Compose, the voices directory must be a bind mount (not an anonymous volume) pointing to a host path that survives container removal.

s2-server exited prematurely on startup

Usually a CUDA error. Confirm your NVIDIA Container Toolkit is installed correctly: docker run --rm --gpus all nvidia/cuda:12.6.3-base-ubuntu24.04 nvidia-smi. If that fails, fix the toolkit before troubleshooting the container.