predxpramad/ai-demo repository overview

This project demonstrates the deployment of a lightweight transformer-based chat model using TinyLlama-1.1B-Chat. The model is integrated with a FastAPI backend to provide simple REST endpoints for text generation. It supports both CPU and GPU (CUDA) inference, automatically utilizing available hardware for optimized performance. The application is fully containerized using Docker, ensuring portability and easy deployment across environments. This project serves as a practical example of combining AI model inference with DevOps practices, making it suitable for building chatbots, AI assistants, and scalable backend services.

Here are simple, clean steps you can copy into your repo:

Got it — Docker Hub needs short, clean, no clutter steps.

Here’s a simple Docker Hub–friendly version:

⁠Steps to Deploy

1. Pull the image

docker pull predxpramad/ai-demo:latest

1. Run container

CPU:

docker run --name custom-name -p 8000:8000 predxpramad/ai-demo

• Give any name for the container in place of custom-name.
  

docker run --gpus all -p 8000:8000 <custom-name>

1. Open API docs

http://localhost:8000/docs

1. Test /chat endpoint

{
  "prompt": "Explain Docker in simple terms"
}