mbari/modelhub repository overview

⁠ModelHub

Code to host and serve select MBARI models so people can try them and evaluate the behavior on their own data. Under the hood it is an inference service for RF-DETR-style object detection and ViT / DINO-style image classification, built with BentoML⁠ and ONNX Runtime. A small web UI (web/) complements the API for hands-on evaluation and metric plots when you publish them.

Class names come from a Hugging Face–style config.json next to each model (id2label / label2id).

The modelhub is live here if you want to try it: https://cortex.shore.mbari.org/modelhub/⁠

Reach out to [email protected]⁠ if you want to add your model. The requirement is to export to onnx format and container and have a config.json in Hugging Face-style.

⁠Layout

• service.py⁠ — BentoML InferenceService with POST /detect, POST /classify, and POST /classify/{model_name} (multipart image field image).
• serving.yaml⁠ — model paths, input sizes, thresholds, ONNX I/O name overrides.
• runtime/⁠ — config loading, preprocessing, ORT runners, label mapping, postprocessing.
• test_images/⁠ — sample assets for tests.
• scripts/test.py⁠ — unittest test suite for both model endpoints.
• web/⁠ — React + Vite UI for trying models and viewing published metric plots.
• bentofile.yaml⁠ — Bento build + GPU-oriented Docker base (nvidia/cuda + cuDNN runtime).
• compose.yaml⁠ — Docker Compose stack (BentoML API + nginx-served web UI).
• pyproject.toml⁠ / uv.lock⁠ — Python dependencies (uv⁠); requirements.txt⁠ is exported for bentofile.yaml⁠ / bentoml build (regenerate after lockfile changes; see below).

⁠Screenshots

Thumbnails below link to the full-resolution images in docs/imgs/⁠.

⁠Architecture

sequenceDiagram
    box rgb(0, 123, 255) Client
        participant C as Client / curl
    end
    box rgb(0, 200, 255) Service
        participant S as service.py InferenceService
    end
    box rgb(0, 210, 230) Runtime
        participant CFG as serving.yaml + runtime/config.py
        participant P as runtime preprocessing
        participant POST as runtime postprocessing + labels
    end
    box rgb(0, 0, 230) Model
        participant M as ONNX model (detect/classify)
    end

    C->>S: POST /detect or /classify(/<model>) + image
    S->>CFG: Load model and endpoint settings
    S->>P: Preprocess image tensor
    P->>M: Run inference
    M-->>POST: Return tensors/logits
    POST-->>S: Build structured result
    S-->>C: JSON response

⁠Setup

Use Python 3.11 or higher. Install uv⁠ (one-time), then from the repository root:

uv sync --all-groups
source .venv/bin/activate   # optional; or use `uv run …` for commands
models/seed_models.sh /mnt/DeepSea-AI/models

--all-groups includes the dev dependency group (e.g. pre-commit). For runtime dependencies only, use uv sync.

After changing dependencies in pyproject.toml, run uv lock, then refresh the pinned export used by BentoML:

uv export --no-dev --no-hashes --no-annotate -o requirements.txt

On Linux with NVIDIA GPUs, the default lock uses onnxruntime-gpu from PyPI. For Microsoft’s CUDA 13 nightly index⁠ instead:

uv pip install --pre --index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-13-nightly/pypi/simple/ onnxruntime-gpu

⁠Configure models

1. Place ONNX files under models/ (or any directory you set as MODEL_ROOT). Expected structure:

   models/
   ├── detector/
   │       └── onnx/
   │           ├── inference_model.onnx
   │           └── config.json              # optional (id2label / label2id)
   └── classifier_a/
   │        ├── model.onnx
   │        └── config.json                  # optional (id2label / label2id)
   └── classifier_b/
   │        ├── model.onnx
   │        └── config.json                  # optional (id2label / label2id)
   

   Copy
2. Optionally place config.json beside each ONNX (same folder) with id2label for readable responses.
3. Edit serving.yaml⁠:
   • Set detection.enabled: true and detection.onnx_path (relative to model_root).
   • Set classification_models.<name> entries (required) to expose POST /classify/{name} endpoints.
   • POST /classify uses the first enabled model in classification_models.
   • Adjust detection.input_height / input_width to match your RF-DETR export (export guide⁠).
   • If ONNX output tensor names differ from boxes / labels / scores, set detection.output_map.
   • Classification: place config.json with id2label next to the ONNX if you want label strings in the JSON response.

Example multi-classifier config:

classification_models:
  classifier_a:
    enabled: true
    onnx_path:  classifier_a/model.onnx
    input_size: 224
    top_k: 5
  classifier_b:
    enabled: true
    onnx_path: classifier_b/model.onnx
    input_size: 224
    top_k: 5

Environment variables (optional):

Copy .env.example⁠ to .env. The justfile⁠ sets dotenv-load so just recipes load .env automatically. For GPU hosts where ONNX reports missing libcublasLt.so.*, add MODELHUB_CUDA_LIB_PATH=/usr/local/cuda/lib64 (adjust to your install) so CUDA libraries are found.

⁠Run locally

Prefer just start (loads .env and applies MODELHUB_CUDA_LIB_PATH). Otherwise:

export BENTOML_HOME="$(pwd)/.bentoml"   # recommended for restricted home dirs
set -a && [ -f .env ] && . ./.env && set +a   # optional: MODEL_ROOT, CUDA library paths
bentoml serve service:InferenceService

Open the docs UI at http://127.0.0.1:3000⁠ (default port 3000).

Tests (with server running):

python scripts/test.py

⁠Docker Compose (full stack)

Docker⁠ with Compose⁠ builds and runs the API (from the repo Dockerfile⁠) and the web UI (production build in web/Dockerfile⁠ behind nginx). ./models and ./serving.yaml are bind-mounted read-only into the API container; put ONNX weights and config.json files under models/ and edit serving.yaml as usual.

From the repository root:

docker compose up --build

Open the UI at http://localhost:8080⁠. Nginx proxies /models, /detect, and /classify to the API on the internal Docker network (same-origin in the browser, so no CORS setup).

GPU (Linux): the default compose.yaml runs without a GPU reservation so the stack starts on hosts without the NVIDIA Container Toolkit⁠ (ONNX Runtime can use CPU). For CUDA inference on a Linux machine with an NVIDIA GPU, uncomment the deploy.resources.reservations.devices block under api in compose.yaml⁠, install the toolkit, and run docker compose up --build again.

Environment: add variables under api.environment in compose.yaml (for example MODELHUB_ORT_PROVIDERS, SERVING_CONFIG) or mount a root .env by extending the Compose file—scripts/env_cuda.sh inside the API image still loads /workspace/.env when present if you bind-mount it.

⁠Notes

• Classification outputs: /classify uses the first enabled model in classification_models; /classify/{model_name} uses classification_models[model_name]. Both use the first ONNX output, flattened to a vector, as class logits unless logits_output is set.
• Detection outputs: The service maps ONNX outputs using detection.output_map. RF-DETR exports may use different names or omit scores; adjust YAML or extend runtime/inference.py⁠ if your graph differs.
• GPU: The service requests resources={"gpu": 1} for BentoML/Kubernetes-style scheduling; ONNX Runtime still falls back to CPU if CUDA is unavailable.
• Docker base: the repo Dockerfile⁠ (used by Compose) is nvidia/cuda:12.6.2-cudnn-runtime-ubuntu22.04. bentofile.yaml⁠ lists a separate base for bentoml build / bentoml containerize (nvidia/cuda:13.2.0-cudnn-runtime-ubuntu22.04). Match CUDA/driver expectations to your hosts when using GPU (NVIDIA Container Toolkit⁠).

⁠Web UI

A React + Vite dashboard in web/⁠ runs alongside the BentoML service to test the models.

⁠Prerequisites

• nvm⁠ — manages Node version
• web/.nvmrc pins Node 22

⁠Start

# 1. Start the BentoML inference service (separate terminal)
just start

# 2. Start the web UI
just web-start

# or 

cd web
nvm install   # one-time if Node 22 is not installed yet
nvm use
cp .env.example .env   # optional — VITE_API_BASE_URL defaults to empty (Vite proxy handles it)
npm install
npm run dev

Open http://localhost:5173⁠.

The Vite dev server proxies /models, /detect, and /classify/* to the BentoML service at http://127.0.0.1:3000 automatically.

To run the API and a production-built UI in containers instead, use Docker Compose (full stack)⁠.

⁠Add sample images

Copy representative images to models/<modelId>/samples/sample_{1,2,3}.png. They pre-populate the thumbnail strip on each model's Try page and serve as the model-card thumbnail on the landing page. These assets are served directly from models/ in both Vite dev and the nginx production container, so they no longer need to be copied into web/dist.

⁠Add metric plots

1. Drop PNG/JPG plots into models/<modelId>/metrics/.
2. Edit (or create) models/<modelId>/metrics/manifest.json:

[
  { "file": "precision_recall.png", "label": "Precision-Recall Curve" },
  { "file": "confusion_matrix.png", "label": "Confusion Matrix" }
]

See models/README.md⁠ for details on model-local assets.

⁠References

• BentoML⁠
• RF-DETR⁠ — ONNX export⁠