Sign inSign up

zyrabitcore/zyrabit-slm

By zyrabitcore

•Updated about 18 hours ago

The Open Platform for Sovereign AI. Run any AI model on your own infrastructure

Image
Security
Machine learning & AI
Monitoring & observability
2

2.8K

zyrabitcore/zyrabit-slm repository overview

Zyrabit SLM

⁠Zyrabit | Own Your Intelligence.

Artificial Intelligence is becoming critical infrastructure.

Critical infrastructure should not be rented.

Zyrabit enables organizations to deploy, govern and evolve AI on their own infrastructure.

Run any model.

On any hardware.

⁠🎥 See it in Action: True Sovereign AI

Curious about what this looks like in practice? We've recorded a hands-on demo showcasing Zyrabit Platform running 100% offline.

Watch the system process voice commands and capture images in real-time without relying on a single external API or internet connection. This is what true local intelligence looks like.

👉 Watch the Offline Demo Here⁠

Today's AI ecosystem is built around APIs.

Organizations rent intelligence from external providers.

This is convenient.

But it also means:

  • Your roadmap depends on someone else.
  • Your costs depend on someone else.
  • Your compliance depends on someone else.
  • Your future depends on someone else.

We believe AI infrastructure should belong to the organization using it.

Zyrabit Platform is the infrastructure layer for Sovereign AI.

It allows enterprises to:

✓ Deploy AI locally

✓ Govern AI usage

✓ Audit every interaction

✓ Connect any model

✓ Switch hardware without rewriting applications

✓ Operate in air-gapped environments

The future of AI infrastructure should be open.

Organizations should never be locked to a single model.

A single cloud.

Or a single vendor.

That's why Zyrabit Platform is open source.

⁠⚡ Quick Start

# Pull the image
docker pull zyrabitcore/zyrabit-slm:2.4.1
⁠Run standalone
docker run -d \
  --name zyrabit-api \
  -p 8088:8080 \
  -e INFERENCE_PROVIDER=ollama \
  -e SLM_URL=http://host.docker.internal:11434 \
  -e MODEL_NAME=qwen2.5:7b \
  -e ZYRABIT_API_KEY_WEB=my-secure-token \
  -v ./documents:/app/document_source \
  -v ./db_data:/app/db_data \
  zyrabitcore/zyrabit-slm:2.4.1
⁠Run the full stack
git clone https://github.com/Zyrabit-tech/zyrabit-SLM.git
cd zyrabit-SLM && chmod +x zyra.sh

./zyra.sh install
⁠Verify
# Health check
curl http://localhost:8088/v1/health

# First query
curl -X POST http://localhost:8088/v1/chat \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer my-secure-token" \
  -d '{"text": "Hello, Zyrabit!"}'

⁠🐳 Docker Compose Profiles

cd zyrabit-SLM/zyrabit-slm
cp example.env .env

# Default stack (API + Web UI + Ollama + ChromaDB + Grafana + Prometheus + MCP)
docker compose up -d

# Production — Traefik TLS proxy with rate limiting
docker compose --profile production up -d

# Tenstorrent NPU hardware acceleration
docker compose --profile tenstorrent up -d

# PostgreSQL instead of SQLite
docker compose --profile db up -d

# n8n workflow automation
docker compose --profile automation up -d

# Headless inference only (no UI, no RAG)
docker compose --profile bare up -d

# Combine profiles
docker compose --profile production --profile db --profile tenstorrent up -d

# Stop everything
docker compose down

⁠📁 Volumes

-v ./document_source:/app/document_source   # Uploaded documents
-v ./db_data:/app/db_data                   # SQLite WAL + FTS5 index
-v ./chroma-data:/data                      # Vector embeddings
-v ./prompts:/app/prompts:ro                # System prompts
-v ~/.ollama:/root/.ollama                  # Model weights

⁠⚙️ Environment Variables

ZYRABIT_API_KEY_WEB=<token>             # API auth (openssl rand -hex 32)
INFERENCE_PROVIDER=ollama               # ollama | ollama_host | openai_compatible
SLM_URL=http://zyrabit-engine:11434     # Inference engine
MODEL_NAME=qwen2.5:7b                   # Model to load
DB_URL=http://zyrabit-db:8000           # ChromaDB
EMBEDDING_MODEL=mxbai-embed-large       # Embedding model
NODE_ENABLE_OCR=false                   # Tesseract OCR for scanned PDFs
NODE_RETRIEVAL_MODE=hybrid              # hybrid | vector | keyword

⁠🏥 Health Checks

docker compose ps                                  # Container status
curl -s http://localhost:8088/v1/health | jq        # API health
docker exec zyrabit-engine ollama list              # Models loaded
curl -s http://localhost:8000/api/v1/heartbeat      # Vector DB

⁠🔧 Operations

./zyra.sh install          # Guided setup & launch
./zyra.sh start            # Start stack
./zyra.sh stop             # Stop stack
./zyra.sh verify           # Health check all services
./zyra.sh benchmark        # Inference throughput (t/s + TTFT)
./zyra.sh audit            # Zero-egress compliance report
./zyra.sh validate         # Full offline QA suite
⁠Logs
docker compose logs -f zyrabit-api       # API logs
docker compose logs -f zyrabit-engine    # Inference logs
docker compose logs --tail=100           # Last 100, all services

⁠Design Principles

Platform over models.

Ownership over subscriptions.

Interoperability over lock-in.

Governance over opacity.

Open standards over proprietary ecosystems.

⁠🛠️ Contribution & Development

Please refer to CONTRIBUTING.md⁠ for branching rules (base branch: beta).

MIT © 2026 Zyrabit⁠ | Enterprise Privacy, Local Intelligence.

Tag summary

Content type

Image

Digest

sha256:1eb6a4f14…

Size

515.9 MB

Last updated

about 18 hours ago

docker pull zyrabitcore/zyrabit-slm:2.4.4-6f7cced