opea/lvm-predictionguard repository overview

⁠LVM Prediction Guard Microservice

Prediction Guard⁠ allows you to utilize hosted open access LLMs, LVMs, and embedding functionality with seamlessly integrated safeguards. In addition to providing a scalable access to open models, Prediction Guard allows you to configure factual consistency checks, toxicity filters, PII filters, and prompt injection blocking. Join the Prediction Guard Discord channel⁠ and request an API key to get started.

Visual Question and Answering is one of the multimodal tasks empowered by LVMs (Large Visual Models). This microservice supports visual Q&A by using a LLaVA model available via the Prediction Guard API. It accepts two inputs: a prompt and an image. It outputs the answer to the prompt about the image.

⁠🚀1. Start Microservice with Python

⁠1.1 Install Requirements

pip install -r requirements.txt

⁠1.2 Start LVM Service

python lvm.py

⁠🚀2. Start Microservice with Docker (Option 2)

⁠2.1 Setup Environment Variables

Setup the following environment variables first

export PREDICTIONGUARD_API_KEY=${your_predictionguard_api_key}

⁠2.1 Build Docker Images

cd ../../..
docker build -t opea/lvm-predictionguard:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/third_parties/predictionguard/src/Dockerfile .

⁠2.2 Start Service

docker run -d --name="lvm-predictionguard" -p 9399:9399 -e PREDICTIONGUARD_API_KEY=$PREDICTIONGUARD_API_KEY opea/lvm-predictionguard:latest

⁠🚀3. Consume LVM Service

curl -X POST http://localhost:9399/v1/lvm \
    -H 'Content-Type: application/json' \
    -d '{
      "image": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAFUlEQVR42mP8/5+hnoEIwDiqkL4KAcT9GO0U4BxoAAAAAElFTkSuQmCC",
      "prompt": "What is this?",
      "max_new_tokens": 30
    }'