premai/prem-operator repository overview

⁠📡 Prem Operator: On premise AI with Kubernetes

⁠🗺 Overview

Deploy AI models and apps to Kubernetes without hitting every pitfall.

⁠🚀 Quick Start

Deploy the operator and chat with Hermes in 2 or 3 steps. Note that you need a NVIDIA GPU with 8GB of RAM.

1. Install the NVIDIA Operator

2. Install the Prem Operator

   $ helm install latest oci://registry-1.docker.io/premai/prem-operator-chart
   

   Copy

3. Deploy the Hermes + Big-AGI example and forward the ports

   $ kubectl apply -f examples/big-agi.yaml
   $ kubectl port-forward services/big-agi-service 3000:3000
   

   Copy

If you browse to localhost:3000 you'll be able to begin chatting as soon as LocalAI downloads the model.

https://github.com/richiejp/prem-operator/assets/988098/0f06b254-a1a0-4ae5-815a-ed84998f5c89⁠

⁠🔗 Useful Links

• Guides
  • 🦜️🔗Langchain⁠
  • 🧩Ingress⁠
  • 🌐Managed Clusters (GCP, AWS)⁠
  • 📜Getting started⁠
  • 📦Deployment⁠
  • 👩‍💻Developer⁠
  • 🧪vLLM⁠
• Topics
  • 🔍Frequently Asked Questions⁠
  • 🤝Contributing⁠
  • 🎗️Issues⁠
  • 🔧Troubleshooting⁠

⁠💌 Contact & Getting Help

We'd love to hear from you! Feel free to create an issue or reach out to us on Prem's Discord⁠ on the operator channel.

⁠📝 License

Operator SDK is under Apache 2.0 license. See the LICENSE⁠ file for details.