Sign inSign up
vacancyd logo

probablyfinedev/vacancyd

By probablyfinedev

•Updated 26 days ago

Lends a GPU box's spare capacity to local AI servers, and refuses borrowed requests when it is busy.

Image
Machine learning & AI
0

192

probablyfinedev/vacancyd repository overview

⁠vacancyd

vacancyd sits in front of Ollama, llama.cpp, ComfyUI, Chatterbox TTS or the supported routes of an OpenAI-compatible server and decides whether each borrowed request gets through. It checks owner policy and the GPU memory, system memory and CPU it samples. When the machine can't lend, it refuses with HTTP 503 and a Retry-After header instead of forwarding the request. Local work that talks to the server's own port is never gated.

This image is for a Linux machine nobody sits at, such as a dedicated GPU box whose inference servers already run in containers. It holds the binary from the signed Linux release archive, runs as an unprivileged user and reads /etc/vacancyd/vacancyd.toml from a read-only mount. Use owner.presence = "headless", set owner.gpu_pressure.enabled = false because GPU attribution is not implemented on Linux, and give it NVIDIA GPUs with the NVIDIA Container Toolkit (--gpus). Release testing for 1.0.0 did not include physical GPU hardware. For a Windows or Linux desktop you also use yourself, install the release package instead.

docker pull docker.io/probablyfinedev/vacancyd:v1.0.0

The same images are on GHCR as ghcr.io/maxfield-allison/vacancyd. Both serve amd64 and arm64 images; the ARM64 build is a preview.

Tags: vX.Y.Z for a release and latest for the newest release. Pin the image digest for an immutable reference. Every image is signed with the project's release key and carries SLSA-format provenance and a CycloneDX SBOM for each platform.

Licensed under Apache-2.0.

Tag summary

Content type

Image

Digest

sha256:ce941e910…

Size

9.3 MB

Last updated

26 days ago

docker pull probablyfinedev/vacancyd