jeffrouyer/llm-locust repository overview

⁠jeffrouyer/llm-locust:latest

A single-container LLM benchmarking stack that bundles:

• FastAPI backend (API server & metrics)
• Built Vite WebUI served from /webui/dist so you can access the UI at the same address as the backend

This image is intended for reproducible local runs and CI workflows where you want a simple, single artifact that contains both the UI and backend.

Quick start (pull + run)

# pull the published image
docker pull jeffrouyer/llm-locust:latest

# run the container and forward port 8089
docker run --rm -p 8089:8089 jeffrouyer/llm-locust:latest

# open the web UI
open http://localhost:8089/   # or curl -fsS http://localhost:8089/ | head -n 8

Persisting model/tokenizer cache (recommended)

To avoid repeated downloads of tokenizer/model artifacts, mount a persistent volume:

docker volume create llm_locust_transformers_cache
docker run --rm -p 8089:8089 \
	-e TRANSFORMERS_CACHE=/cache/transformers \
	-v llm_locust_transformers_cache:/cache/transformers \
	jeffrouyer/llm-locust:latest

Docker Compose (recommended for repeatable runs)

The repository provides a docker-compose.yml that defaults to this image. Run the published image (no local build):

DOCKER_IMAGE=jeffrouyer/llm-locust:latest docker compose up --no-build --force-recreate -d

Configuration & environment variables

• TRANSFORMERS_CACHE — path inside the container to store model/tokenizer cache (default used in compose: /cache/transformers)
• PORT — port mapping used by run helpers (the container exposes 8089)

Build-time / publishing variables (used by the repo's build.sh helper)

• DOCKER_IMAGE — name to tag/push the image (eg. jeffrouyer/llm-locust:latest)
• DOCKER_PUSH — set to true to push when using the helper script
• DOCKERHUB_USERNAME / DOCKERHUB_PASSWORD — credentials for automated pushes
• RELEASE_TAG / AUTO_TAG — tag images with a release identifier or auto-generate from commit SHA

Health & diagnostics

• UI root: http://localhost:8089/⁠
• Metrics endpoint: http://localhost:8089/stats/requests⁠
• Logs: docker logs <container-id>

If the built web UI appears missing, ensure your CI or build step built the web assets — the Dockerfile uses a multi-stage build that compiles the web UI in a separate stage.

CI / Release recommendations

1. Build and push mutiple tags from CI: latest + a release tag (e.g. v1.2.3 or commit-sha). The repository's build.sh supports RELEASE_TAG and AUTO_TAG to help with this.
2. Use docker compose up --no-build in production to run the image from the registry instead of building locally.

Troubleshooting

• If you see warnings about volumes when using docker compose across different directories/projects, create the named volume used by the compose file:

docker volume create llm_locust_transformers_cache

• If you get permission errors writing cache files inside the container, set TRANSFORMERS_CACHE to a writable host directory or change ownership on the mounted volume.

Feel free to copy this file into the Docker Hub README for jeffrouyer/llm-locust — I can also add a GitHub Actions workflow to build/tag/publish images automatically if you'd like.