alpindale/aphrodite-engine repository overview

The docker image for Aphrodite Engine⁠. Configure the engine options using env variables passed to the docker. e.g.:

sudo docker run --gpus '"all"' --shm-size 10g -p 2242:2242 -e MODEL_NAME="mistralai/Mistral-7B-Instruct-v0.2" -e PORT=7860 -it alpindale/aphrodite-engine

This would launch the mistral-7b model at port 7860. See here⁠ for all the available variables, and here⁠ for all you can pass to the CMD_ADDITIONAL_ARGUMENTS variable.