vtdat58/tritonforge repository overview

⁠TritonForge

A fullstack Rust web application for converting ONNX deep learning models into TensorRT engines. It provides a browser-based workflow for model upload, TensorRT configuration, GPU selection, job tracking, log inspection, and output management.

⁠Features

• Upload ONNX files from the browser or select files from a server path
• Select locally available TensorRT Docker images (nvcr.io/nvidia/tensorrt:*)
• Detect and select NVIDIA GPUs via nvidia-smi
• Configure TensorRT options: dynamic shapes, workspace size, FP16, explicit batch
• Track conversion jobs with status, timestamps, and container logs
• Download completed TensorRT engine outputs
• Organize outputs into deployment-oriented model groups
• Serve models with Triton Inference Server after grouping them

⁠Requirements

• NVIDIA GPU with drivers installed
• Docker daemon with NVIDIA Container Toolkit (--gpus support)
• TensorRT Docker images pulled locally (nvcr.io/nvidia/tensorrt:*)
• Triton Inference Server Docker images pulled locally (for serving model groups with Triton Inference Server)

⁠Quick Start

docker run -d \
  --name tritonforge \
  --gpus all \
  -p 8080:8080 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /your/data/path:/your/data/path \
  -e DATABASE_URL=sqlite:///your/data/path/converter.db \
  -e DATA_DIR=/your/data/path \
  -e RUST_LOG=info \
  vtdat58/tritonforge:latest

Then open http://localhost:8080 in your browser.

⁠Important: DATA_DIR Path

TritonForge spawns TensorRT conversion containers as sibling containers on the host Docker daemon. Docker resolves bind-mount paths against the host filesystem, so DATA_DIR must be the same absolute path on both the host and inside the TritonForge container:

-v /your/data/path:/your/data/path   # host path == container path
-e DATA_DIR=/your/data/path

⁠Environment Variables

⁠Source

github.com/dat58/TritonForge⁠