t41372/open-llm-vtuber repository overview

⁠Open-LLM-VTuber

Talk to any LLM with hands-free voice interaction, voice interruption, Live2D taking face, and long-term memory running locally across platforms. Now runs with docker.

Read more about this project on GitHub⁠

⁠Docker

⚠️⚠️⚠️ This is highly experimental with many issues. Some features are not available in the docker version as well.

Current issues:

• Large image size (~20GB), and will require more space because some models are optional.
• Nvidia GPU required (GPU passthrough)
• Nvidia Container Toolkit⁠ needs to be configured for GPU passthrough.
• Some models will have to be downloaded again if you stop the container. (will be fixed)
• Doesn't support Arm CPUs. One of the dependencies (grpc, to be exact) will fail for some reason https://github.com/grpc/grpc/issues/34998⁠.
• And as mentioned before, you can't run it on a remote server unless the web page has https. That's because the web mic on the front end will only launch in a secure context (which means localhost or https environment only).

Most of the asr and tts will be pre-installed. However, bark TTS and the original OpenAI Whisper (Whisper, not WhisperCPP) are NOT included in the default build process because they are huge (~8GB, which makes the whole container about 25GB). In addition, they don't deliver the best performance either. To include bark and/or whisper in the image, add the argument --build-arg INSTALL_ORIGINAL_WHISPER=true --build-arg INSTALL_BARK=true to the image build command.

Setup guide:

1. Review conf.yaml before building (currently burned into the image, I'm sorry):

2. Build the image:

docker build -t open-llm-vtuber .

(Grab a drink, this will take a while)

1. Grab a conf.yaml configuration file. Grab a conf.yaml file from this repo. Or you can get it directly from this link⁠.

2. Run the container:

$(pwd)/conf.yaml should be the path of your conf.yaml file.

docker run -it --net=host --rm -v $(pwd)/conf.yaml:/app/conf.yaml -p 12393:12393 open-llm-vtuber

1. Open localhost:12393 to test