ardanlabs/kronk repository overview

⁠Kronk — Local LLM Inference & Audio Transcription Server

Kronk is a high-performance, GPU-accelerated server for local LLM inference and audio transcription. Built on llama.cpp and whisper.cpp, it provides an OpenAI-compatible API for chat completions, responses, embeddings, reranking, and audio transcription.

⁠Quick Start

Run the server with CPU inference:

docker run --rm -p 11435:11435 -v kronk-data:/kronk ghcr.io/ardanlabs/kronk:latest

Check that it's running:

curl http://localhost:11435/v1/liveness

Models download automatically on first use.

⁠Available Images

⁠Platform Support

Legend:

• ✅ Fully supported with GPU acceleration
• ⚠️ Works with significant caveats

⁠GPU Run Examples

⁠NVIDIA GPU

Requires NVIDIA Container Toolkit⁠.

docker run --rm --gpus all -p 11435:11435 -v kronk-data:/kronk ghcr.io/ardanlabs/kronk:latest

Specific GPUs by index:

docker run --rm --gpus '"device=0,1"' -p 11435:11435 -v kronk-data:/kronk ghcr.io/ardanlabs/kronk:latest

Specific GPUs by UUID:

docker run --rm --gpus '"device=GPU-3a1f...,GPU-9b2e..."' -p 11435:11435 -v kronk-data:/kronk ghcr.io/ardanlabs/kronk:latest

⁠AMD GPU (ROCm)

Requires /dev/kfd and /dev/dri access. The user running Docker must be in the render and video groups.

docker run --rm \
  --device=/dev/kfd --device=/dev/dri \
  --group-add video --group-add render \
  --security-opt seccomp=unconfined \
  -p 11435:11435 -v kronk-data:/kronk \
  ghcr.io/ardanlabs/kronk:rocm

⁠Vulkan (AMD, Intel, NVIDIA)

A single image that works across GPU vendors. Requires Mesa Vulkan drivers on the host.

docker run --rm --device=/dev/dri --group-add video --group-add render \
  -p 11435:11435 -v kronk-data:/kronk \
  ghcr.io/ardanlabs/kronk:vulkan

Vulkan on NVIDIA: Requires NVIDIA Container Toolkit with graphics capabilities:

docker run --rm --gpus all \
  -e NVIDIA_DRIVER_CAPABILITIES=compute,utility,graphics \
  -p 11435:11435 -v kronk-data:/kronk \
  ghcr.io/ardanlabs/kronk:vulkan

Vulkan on Intel: Expose /dev/dri (ANV driver):

docker run --rm --device=/dev/dri --group-add video --group-add render \
  -p 11435:11435 -v kronk-data:/kronk \
  ghcr.io/ardanlabs/kronk:vulkan

Multi-GPU selection: Pin a specific GPU with environment variables:

# Select by PCI ID (format: <vendor>:<device>)
# 1002 = AMD, 8086 = Intel, 10de = NVIDIA
docker run --rm --device=/dev/dri --group-add video --group-add render \
  -e MESA_VK_DEVICE_SELECT='1002:744c' \
  -p 11435:11435 -v kronk-data:/kronk \
  ghcr.io/ardanlabs/kronk:vulkan

Verify Vulkan devices inside the container:

docker exec -it <container> vulkaninfo --summary

⁠NVIDIA Jetson (Orin / Xavier)

Requires JetPack 6+ and nvidia-container-runtime configured as the default runtime.

docker run --rm --runtime nvidia \
  -e NVIDIA_DRIVER_CAPABILITIES=compute,utility,graphics \
  -p 11435:11435 -v kronk-data:/kronk \
  ghcr.io/ardanlabs/kronk:jetson

Verify CUDA detection:

docker run --rm --runtime nvidia \
  --entrypoint nvidia-smi \
  ghcr.io/ardanlabs/kronk:jetson

⁠Audio Transcription (Bucky)

Every image includes whisper.cpp shared libraries and ffmpeg for decoding non-PCM audio uploads (WebM/Opus, MP4/AAC, OGG, M4A).

Pull a Whisper model (models are ~50-500MB, not baked into images):

docker exec -it <container> kronk bucky model pull tiny.en

Transcribe an audio file:

curl -X POST http://localhost:11435/v1/audio/transcriptions \
  -F file=@samples/jfk.wav \
  -F model=tiny.en \
  -F response_format=json

⁠Ports

To keep the debug server local-only, override KRONK_WEB_DEBUG_HOST=127.0.0.1:11445.

⁠Volume & Persistence

Mount a volume on /kronk to persist models, catalog, keys, and libraries across container restarts:

docker run --rm -p 11435:11435 -v kronk-data:/kronk ghcr.io/ardanlabs/kronk:latest

Models, libraries, catalog data, and API keys are all stored under /kronk. Named volumes are recommended — bind mounts require chown 10001:10001 on the host directory (the container runs as UID/GID 10001).

⁠Environment Variables

⁠Verifying Image Signatures

Every published manifest is signed with cosign⁠ keyless (Sigstore / Fulcio / Rekor) by the docker.yml⁠ GitHub Actions workflow. There are no long-lived signing keys — the signature is bound to the workflow identity and recorded in the public Rekor transparency log.

To avoid cluttering the ardanlabs/kronk tag list with sha256-*.sig entries, Docker Hub signatures are stored in a sibling repository, ardanlabs/kronk-signatures⁠. Verifiers must point cosign at that repo via COSIGN_REPOSITORY:

COSIGN_REPOSITORY=ardanlabs/kronk-signatures \
  cosign verify ardanlabs/kronk:latest \
    --certificate-identity-regexp \
      'https://github.com/ardanlabs/kronk/.github/workflows/docker.yml@.*' \
    --certificate-oidc-issuer https://token.actions.githubusercontent.com

The mirror images on GHCR keep signatures co-located (the default cosign layout), so verifying those does not need COSIGN_REPOSITORY:

cosign verify ghcr.io/ardanlabs/kronk:latest \
  --certificate-identity-regexp \
    'https://github.com/ardanlabs/kronk/.github/workflows/docker.yml@.*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Substitute latest with any published tag (<version>-<variant>, latest-<variant>, etc.). A successful verification prints the Fulcio cert, the GitHub Actions workflow identity, and the Rekor log entry index.

⁠Links

• Source: https://github.com/ardanlabs/kronk⁠
• Documentation: https://github.com/ardanlabs/kronk#readme⁠
• Website: https://kronkai.com⁠
• Signatures (Docker Hub): https://hub.docker.com/r/ardanlabs/kronk-signatures⁠