jahnavik186/ai-observability-platform repository overview

⁠🚀 AI Observability Platform

AI Observability Platform is a self-hosted observability solution built for modern AI and LLM-powered applications. It helps developers, researchers, and organizations monitor model latency, token usage, estimated cost, errors, and quality signals in one unified platform. This containerized platform enables teams to run observability locally, integrate monitoring into production workflows, and build reliable, transparent, and cost-aware AI systems without vendor lock-in.

⁠✨ Key Features

• 🤖 Observability for AI and LLM applications
• 📈 Latency monitoring across models, endpoints, and environments
• 💸 Token usage and cost tracking
• ❌ Error and failure visibility
• 🎯 Accuracy scoring and evaluation workflows
• 📊 Grafana dashboards for metrics and trends
• ⚡ Fast local deployment with Docker Compose
• 🌍 Self-hosted and reusable across AI projects
• 🧩 Provider-agnostic design for multiple AI backends

⁠🏗️ Architecture

AI Observability Platform is composed of a few core services that work together:

• SDK / Instrumentation Layer – captures telemetry from your AI application
• Collector API – ingests events, traces, and scores
• PostgreSQL – stores traces, metrics, and scoring history
• Redis – maintains real-time counters and fast lookups
• Grafana – visualizes dashboards for latency, cost, errors, and quality

⁠Architecture Flow

Your AI Application
        |
        |  @observe / score()
        v
   AI Observability SDK
        |
        |  batched telemetry events
        v
   Collector API (FastAPI)
      /        \
     /          \
    v            v
PostgreSQL     Redis
(traces,       (real-time
 scores,        counters,
 history)       fast metrics)
        \        /
         \      /
          v    v
          Grafana
   (dashboards & analytics)

⁠📦 Run with Docker

Pull the image: docker pull jahnavik186/ai-observability-platform

⁠🔗 GitHub repository:

https://github.com/jahnavik186/AI-Observability-Platform⁠

⁠👩‍💻 Maintainer

Jahnavi Kachhia
Global Product Owner, AI & ML | Open Source Contributor