gafda/vidupe-net

By gafda

•Updated 5 months ago

High-performance video duplicate finder.

Image

Content management system

987

Overview Tags

gafda/vidupe-net repository overview

⁠Vidupe.Net

High-performance video duplicate finder — powered by .NET 10, Blazor Server, FFmpeg, and optional GPU acceleration (NVIDIA CUDA · AMD ROCm/OpenCL · Intel OpenCL · CPU fallback).

👤 Developer: @gafda⁠
📄 License: GPL v3

main

⁠What is Vidupe.Net?

Vidupe.Net scans your video library and surfaces near-duplicate files through seven complementary algorithms:

Algorithm	Speed	Accuracy	Description
CLIP	★★☆☆☆	★★★★★+	Semantic visual matching via CLIP ViT-B/32 (ONNX, GPU recommended)
LPIPS	★★☆☆☆	★★★★★	Deep perceptual distance via AlexNet backbone (ONNX, GPU recommended)
SSIM	★★★☆☆	★★★★☆	Structural Similarity Index (GPU-accelerable via ILGPU)
pHash	★★★★★	★★★☆☆	DCT perceptual hash — fast structural fingerprinting
dHash	★★★★★	★★★☆☆	Gradient-based difference hash — robust to scaling
Histogram	★★★★★	★★☆☆☆	Color distribution correlation
MSE	★★★★★	★☆☆☆☆	Mean Squared Error — pixel-level comparison

Results appear in a clean browser UI with side-by-side previews, metadata comparison, and one-click delete or ignore actions. Everything is cached in SQLite so repeated scans are instant.

⁠Image tags

Tags follow semantic versioning⁠ (X.Y.Z).

Tag	Architecture	Description
`latest`	amd64 + arm64	Multi-arch alias for the most recent stable release
`1.8.0`	amd64 + arm64	Current stable release

Both tags are multi-arch manifest lists — Docker and Podman automatically pull the correct image for your host architecture. Use an exact X.Y.Z tag for reproducible deployments; latest is convenient for quick testing.

⁠Origin & Credits

Vidupe.Net is an independent implementation inspired by the original vidupe⁠ created by Kristian Koskimäki.

While this is a completely separate codebase written in .NET, it honors the legacy of the original tool by continuing its mission to provide a powerful, open-source solution for video library management. We are grateful for the conceptual foundation laid by the original project.

⁠Quick start

# Docker
docker run -d \
  --name vidupe \
  -p 8080:8080 \
  -v /path/to/your/videos:/videos:rw \
  -v vidupe-data:/data \
  gafda/vidupe-net:1.8.0

# Podman
podman run -d \
  --name vidupe \
  -p 8080:8080 \
  -v /path/to/your/videos:/videos:rw \
  -v vidupe-data:/data \
  gafda/vidupe-net:1.8.0

Then open http://localhost:8080⁠ in your browser, add /videos as a scan folder, and start scanning.

⁠Docker Compose

⁠Basic (CPU only)

services:
  vidupe:
    image: gafda/vidupe-net:1.8.0
    container_name: vidupe
    ports:
      - "8080:8080"
    volumes:
      # Your video library (read-only is enough for scanning but not for removal)
      - /path/to/videos:/videos:ro
      # Add as many source directories as you need
      # - /mnt/nas/clips:/clips:ro
      # Persistent cache database and settings
      - vidupe-data:/data
    environment:
      - Vidupe__DataDirectory=/data
    restart: unless-stopped

volumes:
  vidupe-data:

⁠GPU Acceleration

Vidupe.Net uses two independent GPU systems — ILGPU (pHash DCT, SSIM) and ONNX Runtime (CLIP, LPIPS). The container image is GPU-agnostic — GPU drivers are injected at runtime by the host toolkit. Add the relevant section to your Compose service and choose only one.

⁠NVIDIA (CUDA)

Docker: Install nvidia-container-toolkit⁠.
Podman: Install nvidia-ctk CDI⁠.

# Docker Compose
services:
  vidupe:
    image: gafda/vidupe-net:1.8.0
    container_name: vidupe
    ports:
      - "8080:8080"
    volumes:
      - /path/to/videos:/videos:ro
      - vidupe-data:/data
    environment:
      - Vidupe__DataDirectory=/data
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

volumes:
  vidupe-data:

# Podman (CDI-based, no compose changes needed)
podman run -d --name vidupe -p 8080:8080 \
  --device nvidia.com/gpu=all \
  -v /path/to/videos:/videos:ro \
  -v vidupe-data:/data \
  gafda/vidupe-net:1.8.0

Verify GPU access:

# Docker
docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi
# Podman
podman run --rm --device nvidia.com/gpu=all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi

⁠AMD (ROCm / OpenCL)

Prerequisite: AMDGPU kernel module and ROCm runtime on the host.
Desktop cards RX 5000+ work with the open-source amdgpu driver.

services:
  vidupe:
    image: gafda/vidupe-net:1.8.0
    container_name: vidupe
    ports:
      - "8080:8080"
    volumes:
      - /path/to/videos:/videos:ro
      - vidupe-data:/data
    devices:
      - /dev/kfd:/dev/kfd
      - /dev/dri:/dev/dri
    group_add:
      - video
      - render
    environment:
      - Vidupe__DataDirectory=/data
    restart: unless-stopped

volumes:
  vidupe-data:

# Podman equivalent
podman run -d --name vidupe -p 8080:8080 \
  --device /dev/kfd --device /dev/dri \
  --group-add video --group-add render \
  -v /path/to/videos:/videos:ro \
  -v vidupe-data:/data \
  gafda/vidupe-net:1.8.0

⁠Intel (Arc / Iris / integrated graphics, OpenCL)

Prerequisite: i915 or xe kernel driver and Intel compute runtime (intel-opencl-icd) on the host.

services:
  vidupe:
    image: gafda/vidupe-net:1.8.0
    container_name: vidupe
    ports:
      - "8080:8080"
    volumes:
      - /path/to/videos:/videos:ro
      - vidupe-data:/data
    devices:
      - /dev/dri:/dev/dri
    group_add:
      - video
      - render
    environment:
      - Vidupe__DataDirectory=/data
    restart: unless-stopped

volumes:
  vidupe-data:

# Podman equivalent
podman run -d --name vidupe -p 8080:8080 \
  --device /dev/dri \
  --group-add video --group-add render \
  -v /path/to/videos:/videos:ro \
  -v vidupe-data:/data \
  gafda/vidupe-net:1.8.0

⁠Force CPU mode (disable GPU)

Set Vidupe__PreferCpu=true to disable GPU detection entirely:

environment:
  - Vidupe__DataDirectory=/data
  - Vidupe__PreferCpu=true

⁠Volumes

Path	Purpose	Notes
`/data`	SQLite cache DB + settings	Must be persistent — mount a named volume or host path
`/videos` (or any path)	Video source directories	Read-only (`:ro`) is sufficient for scanning; add as many as needed. Read-Write (`:rw`) if intended to delete files.

# Named volume (recommended)
-v vidupe-data:/data

# Host path (alternative)
-v /home/user/.vidupe:/data

⁠Environment variables

Variable	Default	Description
`Vidupe__DataDirectory`	`/data`	Directory for the SQLite cache and settings file
`Vidupe__PreferCpu`	`false`	Set to `true` to skip GPU detection and use CPU only

⁠Accessing over the network

By default Vidupe.Net listens on port 8080. Map it to any host port:

# Access on port 80
-p 80:8080

# Bind only to localhost (reverse-proxy setup)
-p 127.0.0.1:8080:8080

For HTTPS, place a reverse proxy (e.g. Caddy, Nginx, Traefik) in front and forward requests to port 8080.

⁠Mounting network shares (NAS / SMB / NFS)

Pre-mount the share on the host, then bind-mount the directory into the container:

# Mount a Samba share on the host first
sudo mount -t cifs //192.168.1.100/videos /mnt/nas-videos \
  -o username=user,password=pass,vers=3.0

# Then reference the host mount in Docker Compose
volumes:
  - /mnt/nas-videos:/nas-videos:ro

Alternatively, use Docker named volumes with the local driver:

volumes:
  nas-videos:
    driver: local
    driver_opts:
      type: cifs
      o: "username=user,password=pass,vers=3.0"
      device: "//192.168.1.100/videos"

⁠Upgrading

# Docker
docker pull gafda/vidupe-net:1.8.0
docker compose up -d

# Podman
podman pull gafda/vidupe-net:1.8.0
podman stop vidupe && podman rm vidupe
# Re-run with same volumes (data is preserved)

The SQLite cache and settings in /data are forward-compatible between minor versions. A fresh scan after upgrading reuses all cached metadata.

⁠Troubleshooting

Symptom	Solution
Page does not load	Check `docker logs vidupe` — FFmpeg or port conflict
GPU not detected	Verify host drivers and toolkit, then check logs for ILGPU/ONNX output
CLIP/LPIPS on CPU despite GPU	ONNX Runtime CUDA needs CUDA Toolkit 12.x + cuDNN 9.x on the host (injected via nvidia-ctk)
Scan finds no videos	Confirm the volume mount path is correct with `docker exec vidupe ls /videos`
Volumes empty on Podman + SELinux (Fedora, Bazzite, RHEL)	Add `:z` to each `-v` flag to relabel the mount: `-v /path/to/videos:/videos:ro,z`
Very slow first scan	Expected — subsequent scans use cache and are near-instant
Out of memory on large libraries	Raise the container memory limit; the app will auto-pause and resume

# Stream live logs
docker logs -f vidupe
podman logs -f vidupe

# Open a shell inside the container
docker exec -it vidupe /bin/bash
podman exec -it vidupe /bin/bash

⁠Architecture and technical notes

Runtime: .NET 10, published as a self-contained single-file binary
UI: Blazor Interactive Server (WebSocket-based, no npm/webpack)
Cache: SQLite in WAL mode — safe for concurrent reads across scans
GPU (ILGPU): CUDA → OpenCL → CPU fallback for pHash DCT and SSIM
GPU (ONNX Runtime): CUDA → ROCm → CPU fallback for CLIP and LPIPS neural networks
FFmpeg: Bundled via the ffmpeg apt package — supports all common codecs
Container: GPU-agnostic image; NVIDIA/AMD/Intel drivers injected at runtime via host toolkit
pHash Pre-filter: Optional per-position pHash pre-filter with adjustable threshold (0–64 bits) rejects dissimilar pairs before the selected algorithm runs. Enable or disable in Advanced Settings

Vidupe.Net — find the duplicates, reclaim the space.

Tag summary

Recent tags

Content type

Image

Digest

sha256:1b6d256d4…

Size

809.1 MB

Last updated

5 months ago

docker pull gafda/vidupe-net