offlinai/offlinai-server repository overview

⁠OfflinAI Server

Your own private AI server. Run chat, vision, embeddings, reranking, speech-to-text, text-to-speech, image generation and moderation models on your own hardware, behind the API your software already speaks. Nothing leaves your network.

• Open standards: OpenAI's API by default (/v1/chat/completions, /v1/responses, /v1/embeddings, /v1/audio/*, /v1/images/*, /v1/moderations, /v1/batches), plus Ollama's API and Anthropic's Messages API.
• Any model: GGUF and Ollama models, or the built-in catalog of downloads.
• Ready for IT: API keys with limits and budgets, organisations and single sign-on, a signed audit log, Prometheus metrics and OpenTelemetry traces, a web console in 17 languages.
• Works air-gapped, including licensing.

Product page: https://offlinai.com/offlinai-server.html⁠

⁠Quick start (Free plan)

The Free plan serves programs on the same machine. On Linux:

docker run -d --name oas --network host -v oas-data:/data offlinai/offlinai-server

# a one-time link to set the console's password
docker exec oas offlinai-server admin bootstrap --url http://127.0.0.1:12436

Then point any OpenAI client at http://127.0.0.1:12436/v1, and open the console to download models.

⁠Serving other machines (licensed)

With a licence that includes network serving, other machines use HTTPS on port 12443:

docker run --rm -v oas-data:/data offlinai/offlinai-server license activate <key>
docker run -d --name oas -p 12443:12443 -v oas-data:/data offlinai/offlinai-server
docker exec oas offlinai-server admin bootstrap --url https://<host>:12443
docker exec oas cat /data/tls/ca.pem      # the certificate authority clients install

⁠Notes

• Runs as a non-root user; all data (models, keys, usage, audit log, licence, certificates) lives in the /data volume.
• CPU by default. For a GPU, expose it to the container and set -e OAE_BACKEND=vulkan -e OAE_IMAGE_BACKEND=vulkan.
• Tags: each version (0.1.0, …) and latest. Linux x86-64.
• Licensed users find the full administrator and developer guide in the server's web console, under Guide.
• Licence: the OfflinAI Server Licence⁠ — the Free plan is free for personal and internal use; paid plans need a licence key. The licence and third-party notices are in /usr/share/doc/offlinai-server inside the image.