This repo will contain the LiteLlm backend for Indian Constitution & Law from daslearning.in
354
This AI API is designed to be private and can answer your queries on Indian Constitution & Indian Penal Code (IPC). This app is LiteLlm wrapper which is build on Ollama LLMs which are namely llama3.2 & mxbai-embed-large.
Ollama backend on a container and make sure nvidia-container-toolkit is installed if you have Nvidia GPU. The below image will automatically download the required models during startup which may take some time to be ready, depending on internet speed.docker pull sdas92/ollama-llama3.2-mxbai-embed:v1 # check for the latest version at docker hub
# for GPU machines
docker run -d --gpus=all -p 11434:11434 --name ollama sdas92/ollama-llama3.2-mxbai-embed:v1
# for CPU only machines
docker run -d -p 11434:11434 --name ollama sdas92/ollama-llama3.2-mxbai-embed:v1
LiteLlm backend (which can also be accessed via API calls). In the Ollama URL, use the container IP, do not use localhost as it will try access the ollama within litellm containerdocker pull sdas92/law-litellm:v1
docker run -d -p 4000:4000 -e OLLAMA_API_BASE="http://your-ollama-host:11434" --name litellm-rag sdas92/law-litellm:v1 # use the container IP
# You can test or use the API direcly with your application, sample format
curl -X POST http://localhost:4000/chat/completions -H 'Content-Type: application/json' -H 'Authorization: Bearer sk-1234' -d '{"model": "indian-law-llm", "messages": [{"role": "user", "content": "Punishment for money fraud?"}]}'
Content type
Image
Digest
sha256:8769c6369…
Size
280 MB
Last updated
over 1 year ago
docker pull sdas92/law-litellm:v1