mallison01/photo-pipeline repository overview

⁠photo-pipeline

A self-hosted container that watches an "incoming" photo and video folder, sorts everything into a dated library, fixes the timestamps and GPS that phones and drones get wrong, tells Immich⁠ to rescan, and gives you a web page to run and configure all of it.

One image, one container, one web UI on port 8420. Everything it keeps (configuration, databases, logs, backups) lives in a single volume you mount.

⁠What it does

• Organizes by capture date. New files in your incoming ("Source") folder are moved into your library ("Destination"), by default under Year / Year_Month / Year_Month_Day, using the date the photo or video was really taken (falling back to the file date). Runs when files arrive, on a schedule, or when you click Run Now.
• Dry Run. Shows exactly what a run would do, as a report, without moving, renaming or deleting anything.
• Fixes wrong timestamps and missing GPS. Per-device profiles for cameras and drones whose clocks or time zones are off (including pairing a drone's video with its goggles' footage), and optional GPS backfill from a Dawarich⁠ location history.
• Immich integration (optional). Triggers an external-library scan after a run and puts anything needing a human look into a review album.
• Housekeeping. Removes empty folders left behind, deletes junk files by name from the incoming folder (desktop.ini and the like, from a list you edit), and clears thumbnail caches.
• Schedules. Run at a set time on chosen days, or repeat every N minutes or hours.
• A real web UI. Pipelines and their configuration, run history, reports and insights, users and roles, email notifications (SMTP), backups (scheduled, on demand, and restore), plus two standalone tools, a Media Mover and a Metadata Editor.
• Knows its own version, and whether a newer one exists. The sidebar shows the running release; Settings → About lists every real installed component version and checks Docker Hub's own public tags once a day (plus an on-demand Check Now) for a newer stable release — informational only, it never pulls or applies anything by itself.

⁠Tags

To see exactly what a container was built from:

docker exec photo-pipeline cat /app/BUILD_INFO

⁠Quick start

Create two folders on the host: one for the app's own data, and your media root (the folder that contains both your incoming folder and your library).

services:
  photo-pipeline:
    image: mallison01/photo-pipeline:stable
    container_name: photo-pipeline
    restart: unless-stopped
    init: true
    ports:
      - "8420:8420"
    environment:
      ADMIN_USERNAME: "admin"
      ADMIN_PASSWORD: ""      # blank: a random one is generated and printed to the logs once
      PUID: "1000"            # the user that should own files this container writes
      PGID: "1000"
    volumes:
      - /path/to/photo-pipeline-data:/data    # all app state: config, databases, logs, backups
      - /path/to/your/media:/media            # your library root, read/write

mkdir -p /path/to/photo-pipeline-data     # Docker will not create a missing host folder
docker compose up -d
docker compose logs -f                    # first boot prints the admin password, once

Open http://<host>:8420, sign in, and set your own password. Then:

1. Settings → Pipelines → create a pipeline and pick its Source (incoming) and Destination (library) folders with the folder browser. Everything you mounted at /media is browsable.
2. Optionally fill in Immich and Dawarich on the pipeline's Configuration page, then click Test Configuration. It checks the servers, the API keys' permissions and every folder for real, and tells you what is wrong before a real run does.
3. Click Dry Run, read the report, then Run Now (or let the file watcher and your schedules take over).

The first start needs outbound internet access once, to install a time-zone helper into the data volume. It is kept across restarts and updates.

Equivalent docker run:

docker run -d --name photo-pipeline --init --restart unless-stopped \
  -p 8420:8420 \
  -e ADMIN_USERNAME=admin -e PUID=1000 -e PGID=1000 \
  -v /path/to/photo-pipeline-data:/data \
  -v /path/to/your/media:/media \
  mallison01/photo-pipeline:stable

⁠Configuration

Everything else (Immich and Dawarich addresses and keys, schedules, time zone, SMTP, user accounts, login lockout, session length) is configured in the web UI, not through environment variables.

⁠Synology

Three things are specific to a Synology NAS:

1. PUID / PGID. Read them from the folder that owns your media:

   ls -ln /volume1/Shares/YourShare
   

   Copy
   The two numbers after the permission string are the owner and group.
2. EXTRA_GROUPS. If the app's folder browser shows nothing for folders you know exist, your share's ACL probably names a group instead of matching your PGID. Look at the ACL, then look the group up:

   /usr/syno/bin/synoacltool -get "/volume1/Shares/YourShare"
   grep '^administrators:' /etc/group
   

   Copy
   Put that group's number in EXTRA_GROUPS (on DSM, administrators is usually 101).
3. Mount DSM's own tools, read-only, so permissions are restored. After it rewrites a file's metadata or moves it, the pipeline copies the destination folder's Synology ACL onto the file, using DSM's synoacltool. That needs these three mounts under volumes::

      - /usr/syno:/usr/syno:ro
      - /usr/lib:/host-lib:ro
      - /lib:/host-lib2:ro
   

   Copy
   Without them everything still runs, but each file logs acl_restore_failed in Reports, and other accounts (File Station, SMB) may not get the folder's permissions on files the pipeline moved. Skip these on a non-Synology host.

⁠Updating

Nothing updates itself. To move to a newer release:

docker compose pull
docker compose up -d

Your data volume carries over, and the new version upgrades its own databases and configuration on start. On the first start of a different build, the container takes a full backup before it changes anything. The Backups page then names that archive, so you always have a restore point.

Rolling back: pin the previous tag (for example image: mallison01/photo-pipeline:v1.1.0) and docker compose up -d. Updates so far only add to your data, which an older version simply ignores, so this is usually enough. If it is not, restore the pre-upgrade backup from Settings → Backups → Restore.

⁠Backups

Settings → Backups makes a single archive of every pipeline's configuration, databases and logs, the app's own accounts database and its settings. You can back up on demand, on a schedule (chosen days, at a set time or repeating), download the file, and restore it, whole or in parts, on this or another machine. Backups are stored in /data/backups by default; point them at another volume so a single disk failure can't take the data and its backups together. The page reminds you until you do.

Backup archives contain your pipeline configuration, including any Immich and Dawarich API keys and the SMTP password. Store them accordingly.

⁠Security notes

• The UI is plain HTTP on port 8420. For anything beyond your own network, put a TLS-terminating reverse proxy in front of it (and set TRUSTED_PROXY_HOPS=1 and FORCE_INSECURE_COOKIES=0).
• Logins are rate-limited and accounts lock after repeated failures. Users have roles, and access to pipelines and folders can be limited per role.
• The file watcher runs as root inside the container, because restoring Synology ACLs requires it; the web application runs as your PUID/PGID.
• API keys are stored in the pipeline's configuration file in /data. Protect that volume.

⁠Requirements and limits

• Docker with Compose v2, or plain docker run.
• A writable host folder for /data, and your media folder for /media.
• Outbound internet on the first start of a fresh data volume.
• Immich and Dawarich are optional. Give the Immich API key only the access the pipeline needs: scan the library, read and create albums, add assets to an album, browse assets by folder, and search asset metadata. It never uploads or deletes photos. Test Configuration exercises exactly those operations, using a throwaway album it creates and then deletes; if the key can't delete albums, it reports that cleanup step as failed and asks you to remove the test album by hand. The pipeline itself never needs that permission.

The application source is developed privately; this image is what is published. The exact commits an image was built from are in /app/BUILD_INFO.