tensorzero/gateway repository overview

⁠TensorZero

TensorZero creates a feedback loop for optimizing LLM applications — turning production data into smarter, faster, and cheaper models.

1. Integrate our model gateway
2. Send metrics or feedback
3. Optimize prompts, models, and inference strategies
4. Watch your LLMs improve over time

It provides a data & learning flywheel for LLMs by unifying:

• Inference: one API for all LLMs, with <1ms P99 overhead
• Observability: inference & feedback → your database
• Optimization: from prompts to fine-tuning and RL
• Evaluations: compare prompts, models, inference strategies
• Experimentation: built-in A/B testing, routing, fallbacks

Website⁠ · Github⁠ · Docs⁠ · Twitter⁠ · Slack⁠ · Discord⁠

Quick Start (5min)⁠ · Tutorial⁠ · Deployment Guide⁠ · API Reference⁠ · Configuration Reference⁠