Lightweight ML model serving (ONNX, PyTorch, scikit-learn)
1.9K
Lightweight ML model serving with ONNX, PyTorch, and scikit-learn support.
docker run -p 8080:8080 \
-v $(pwd)/models:/models \
ogulcanaydogan/model-server
# Run prediction
curl -X POST http://localhost:8080/api/predict \
-H "Content-Type: application/json" \
-d '{"input": [[1.0, 2.0, 3.0, 4.0]]}'
Run inference on input data.
{"input": [[1.0, 2.0, 3.0, 4.0]]}
Response:
{
"output": [[0.95, 0.05]],
"latency_ms": 2.34,
"model": "/models/model.onnx"
}
Get model metadata and configuration.
Health check.
Readiness check (returns 503 if model not loaded).
| Variable | Description | Default |
|---|---|---|
PORT | Server port | 8080 |
MODEL_PATH | Directory containing model files | /models |
MODEL_NAME | Specific model filename | Auto-detect |
RUNTIME | Runtime: onnx, pytorch, sklearn | onnx |
MAX_BATCH_SIZE | Maximum batch size | 32 |
HOT_RELOAD | Auto-reload model on file change | false |
RELOAD_INTERVAL | Reload check interval (seconds) | 60 |
services:
model-server:
image: ogulcanaydogan/model-server
ports:
- "8080:8080"
volumes:
- ./models:/models
environment:
- RUNTIME=onnx
- HOT_RELOAD=true
MIT
Content type
Image
Digest
sha256:90bb40558…
Size
103.1 MB
Last updated
7 months ago
docker pull ogulcanaydogan/model-server