cavo789/media-dedup repository overview

⁠media-hygiene

Hygiene for photo and video libraries spread over several folders and disks: clear out duplicates, spot broken files, then sort what remains into folders you approve. Every change is journaled and can be undone. Everything runs locally, in this image, from PowerShell or WSL.

Source code, full documentation (English and French) and issues: github.com/cavo789/media-hygiene⁠

⁠Quick start

docker run --rm -it -v "C:\Photos:/data/c/Photos:ro" cavo789/media-hygiene audit

This lists the duplicate and broken photos and videos of C:\Photos and changes nothing: :ro (read-only) makes Docker itself forbid any write. Your first audit⁠ explains the command and its result, part by part.

⁠Clean

• Exact duplicates, across folders and disks: same size and same SHA-256, compared byte for byte again right before any deletion.
• Near duplicates and burst series: resized or recompressed copies, re-encoded videos, and bursts whose best shots you choose in your browser, with the keyboard. Near duplicates are moved to the quarantine, never deleted.
• Broken files: empty files, images and RAW files that cannot be decoded, videos that cannot be opened.
• The right copy stays: a folder you protect, the copy with its sidecar (.xmp, .aae), a name someone chose rather than IMG_1234, a named folder rather than DCIM.
• Any file type on request: --ext video, --ext pdf,docx or a category of your own; copies of documents are moved to the quarantine, never deleted.

⁠Sort (in development)

• classify proposes a folder structure from dates, folder names, events, trips and places (GPS resolved offline, optionally a local vision model through Ollama), in an Excel workbook you edit. Nothing is imposed: you change what you want.
• sort applies the edited workbook, journaled, and proves that no file was lost. The next classify keeps your edits.
• Albums as hard links (no copy, no move), a map page to name your places, an inventory of the library exported to Excel.

⁠Safe by design

• A write-ahead journal of every change; undo restores deleted copies from the kept one and quarantined files from the quarantine; history and purge to review and empty it.
• clean refuses to run without a persistent /journal, or on read-only data mounts.
• Protected folders are never modified; excluded folders are never analysed.
• Local: no account, no cloud. Your photos, and where they were taken, never leave your computer. How your photos stay safe⁠.

⁠Mount points

Details: mount points⁠.

⁠A full clean

docker run --rm -it `
  -v "C:\Photos:/data/c/Photos" `
  -v "D:\Old disk:/data/d/Old disk" `
  -v media-hygiene-cache:/cache `
  -v "$HOME\media-hygiene\reports:/reports" `
  -v "$HOME\media-hygiene\config:/config" `
  -v "$HOME\media-hygiene\journal:/journal" `
  -v "$HOME\media-hygiene\quarantine:/quarantine" `
  cavo789/media-hygiene clean

It shows what it will do and asks for confirmation first. Step by step: the user guide⁠ · guide en français⁠.

⁠Tags

• latest: the last release; X.Y.Z: a given release.
• linux/amd64 and linux/arm64, with SBOM and provenance attestations.

⁠License

media-hygiene is released under the MIT license⁠. The image also contains third-party components under their own licenses, among them FFmpeg's ffprobe (LGPL-2.1+, built without GPL parts), libheif and libde265 (LGPL-3.0), x265 (GPL-2.0+) and LibRaw (LGPL-2.1 / CDDL-1.0).