drumsergio/media-triage repository overview

⁠media-triage

Automated media pipeline that monitors a Telegram group for torrent tracker posts, parses release metadata, and presents everything in a review-ready web dashboard.

⁠Features

• Telegram userbot listener (Telethon) for real-time ingestion
• Regex parser that handles tracker cards, HDO posts, and scene-style release names
• Auto-approval: clean items download by themselves, only the doubtful ones wait for you
• Web UI for reviewing, accepting, and rejecting items (FastAPI + Jinja2)
• PostgreSQL storage with async SQLAlchemy
• Docker-ready with multi-arch support (amd64/arm64)

⁠Auto-approval

Most posts are fine, and stopping to approve each one is the slow part. Set AUTO_APPROVE_ENABLED=true and the service decides for itself, keeping only the items worth thinking about.

The rule is that review is for what a check flags, plus duplicates. Missing data alone does not send an item to review. A film without its parental guide waits for it, and comes back to you only if it never arrives. A book no catalogue can classify downloads, unless its own title or author contains a word from the politics, sex or magic lists.

⁠Films and series

First the service has to be sure it is looking at the right title. TMDB is searched with the parsed title and year, and the record that comes back is only accepted when the parsed title is at least 85% alike the record's title or its original title, accents and punctuation ignored, and the years agree within one. A record found only by dropping the year is never accepted: it names a title, not necessarily this one. Everything below reads that record's data, including the IMDb id the parental guide is fetched with, so a wrong match would judge one film on another film's content.

With the match confirmed, a film or series downloads by itself when all of this holds:

• the IMDb parental guide came back;
• sex and nudity is rated none or mild, backed by at least MIN_GUIDE_VOTES votes;
• violence is rated none, mild or moderate, backed by at least MIN_GUIDE_VOTES votes;
• frightening is rated none, mild or moderate, backed by at least MIN_GUIDE_VOTES votes;
• no category has 40% or more of its votes at or above its blocking line, whatever label won;
• TMDB returned keywords, and none of them are magical or supernatural;
• the film is not filed under the Fantasy genre;
• there is no adult certification (US NC-17/X, ES X/18, GB R18/18).

The 40% rule exists because the winning label hides a split vote. A category labelled Mild can have nearly half its voters calling it Moderate or worse, and IMDb publishes the breakdown.

Trip one of those rules and the item goes to the review queue with the reason written out. Miss the data instead, and the item waits. It is re-checked once an hour and handed back to you after AUTO_APPROVE_MAX_WAIT_HOURS with the reason "no parental guide data after N hours". Missing data never counts as clean: an unrated category cannot pass a threshold, and an empty keyword list is silence, not a clean bill of health.

A match that cannot be confirmed goes straight to the review queue saying "could not confirm this is the same title". It never waits, because waiting would not make a wrong match right.

⁠Magazines

Always approved, and a post is only a magazine when it carries the written header the group puts on one. An emoji never decides a type on its own, and a post naming a video quality and a source is a film or a series whatever else it mentions.

⁠Books, audiobooks and comics

Off by default. The catalogues have almost nothing for the Spanish titles this group posts, which leaves the decision resting on one coarse Google Books category, and "Fiction" rules nothing out.

Add them to AUTO_APPROVE_TYPES and the title is looked up on Open Library⁠, with Google Books⁠ as a second opinion, and classified by the subjects those catalogues carry. A record counts when the title and the author both match it. If its subjects mention politics, sex or magic, the book goes to review. If it carries at least three subjects and none of them do, the book downloads.

Most of this group's books get no usable record at all, so a missing classification is not a reason to ask. The post's own title and author are read against the same politics, sex and magic word lists instead. A flagged word sends the book to review with a reason like "title: magic: tarot". Otherwise it downloads, and the reasons say why the catalogue was silent, for example "no catalogue classification (lookup failed: http 429)".

⁠Music, unknown types and duplicates

Always reviewed by hand. The duplicate check runs again at re-check time, so a title approved by hand in the meantime is not downloaded twice.

⁠Re-running the review queue

When the rules change, the review queue still holds items decided under the old ones. Send /rerun to the bot, or run python -m media_triage rerun, and the service decides again every item that went to review because auto-approval was off, its type was not on the list, or no catalogue could classify it. An item you have already acted on is never touched. Neither is one a check flagged, a duplicate, or one whose title match could not be confirmed. The reply counts how many were re-queued, approved, sent back to review and left waiting.

⁠Auditing it

Every automatic decision is written onto the item, with the reasons and the time. The web UI and the Telegram bot both show it.

An item can be rejected right up to the moment the download finishes, and while it is being content-checked. Rejecting it drops the torrent only when this item is what added it: qBittorrent answers an add with the hash of a torrent it was already holding, and deleting that would take a library copy that was there beforehand. The download row is marked cancelled either way, so nothing brings the item back as completed later.

⁠Messages that are not media posts

Notes and links to Instagram, TikTok, YouTube, X or Facebook land in the same group. They are marked ignored and then left alone for good: never reviewed, never re-parsed, never deleted from the group.

That status is deliberately hard to earn. A request typed in plain words is not a release post either, and it stays in the review queue where you can act on it. A bare title in capitals and a sentence saying which book you want are both in this group's history. A message is only finished with when every link in it points somewhere nothing is downloaded from, or when there is nothing in it that could be the name of anything: no capital where grammar does not need one, no year, no shouted word, no quoted phrase, no word like "peli" or "libro", and no line short enough to be a title.

⁠Settings

Defaults live in .env.example⁠.

⁠Database migrations

The schema is managed with Alembic. The service runs alembic upgrade head on start-up, so a new image brings its own columns with it. The migrations in alembic/versions/⁠ only ever add things and are safe to re-run.

⁠Rolling an image back

The migration adds enum labels, and an older image cannot load a label it has never heard of. Move the rows off the new labels before starting the old image:

UPDATE media_items SET status = 'PENDING_REVIEW'
 WHERE status IN ('AWAITING_DATA', 'IGNORED');
UPDATE downloads SET status = 'ERROR' WHERE status = 'CANCELLED';

The columns themselves can stay: the old code simply does not read them.