paragekbote/phi4-mini-reasoning repository overview

⁠phi4-mini-reasoning containerized

This Docker image uses Cog⁠ to package unsloth/Phi-4-mini-reasoning-unsloth-bnb-4bit⁠ model providing a lightning-fast inference experience with minimal VRAM usage.

⁠Model Details

• Base model: unsloth/Phi-4-mini-reasoning-unsloth-bnb-4bit⁠
• Library: Unsloth⁠

⁠Image Purpose

This container:

• ✅ Loads and prepares a 4-bit quantized Phi-4-mini-reasoning.
• ✅ Uses Unsloth + Flash Attention 2 for ultra-fast inference.
• ✅ Is production-ready for GPU-backed deployments using Docker or Cog.

⁠🚀 Usage

1. Pull the image:

docker pull paragekbote/phi4-mini-reasoning:v0.1.0-torch-inference

1. Run the container:

a. Using Docker

docker run -d -p 5001:5000 --gpus all paragekbote/phi4-mini-reasoning:v0.1.0-torch-inference

Send a curl request

curl -X POST http://localhost:5001/predictions \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Summarize Harry Potter in simple terms"}'

b. Using Cog

cog predict \
  -i prompt="Summarize Harry Potter in simple terms" \

⁠📝 Notes

• Designed for inference on consumer GPUs.
• Avoids the need for complex compilation flags or manual setup,it just works.