icsy7867/outetts-api repository overview

NOTE Llama-cpp-python 0.3.7 is now working. Though, on my older Quadrio P6000, 0.3.5 is faster for some reason...

Openai compatible api wrapper for:

https://github.com/edwko/OuteTTS⁠

Your linux host needs the nvidia container toolkit and drivers.

You need to download your desired GGUF and mount in your docker command. -v ./OuteTTS-0.3-1B-Q5_K_M.gguf:/app/model.gguf

If you would like to have your own speakers, mount a folder for /app/speakers into the container. This should have your generated speaker-name.json files. To call this speaker file via the api, called only the file name without the extension. For example, if your speaker file name is person.json, simply use 'person' for the voice.

An example depployment:

docker run -itd --name outetts \
        --device nvidia.com/gpu=all \
        -p 8000:8000 \
        -v ./speakers:/app/speakers:z \
        -v ./app/OuteTTS-0.3-1B-Q5_K_M.gguf:/app/model.gguf:z \
        icsy7867/outetts-api:llamacpp-0.3.6

API information can be found here: http://localhost:8000/docs

NOTES Llama-cpp-python 0.3.7 causes the OuteTTS library to segfault.

For silly tavern, or kobold you can use the the openai compatible. For example: http://192.168.2.7:8000/api/extra/tts

An example curl request:

curl -X 'POST' \
  'http://192.168.2.7:8000/api/extra/tts' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "model": "Outetts",
  "input": "string",
  "voice": "string",
  "voice_characteristics": "string",
  "temperature": "0.2",
  "repetition_penalty": "1.1",
  "max_length": "4096"
}'

NEXT

• Add a function to automatically create speakers from input wav/mp3 files.
• add env variables to automatically download GGUF models from https://huggingface.co/OuteAI/OuteTTS-0.3-1B-GGUF/tree/main⁠
• Currently only works with the 1B model. But will add an automatic method to use the correct tokenizer depending on which model is added.
• env vars to specify default temperature, repetition pen and context length variables.