lucifix/newsnest repository overview

⁠NewsNest

A self-hosted news feed that learns what you care about.

NewsNest gathers your RSS feeds into one calm stream, folds the same story from different outlets into a single card, and uses Claude to rank and summarise what matters to you.

Website⁠ · Features⁠ · Quick start⁠ · How it works⁠ · Configuration⁠ · Development⁠

Feed, ranked for you	Swipe to sort	Dark mode, automatically

_{Outlets and headlines are made up. Photos from Unsplash⁠ by Mohammad Rahmani⁠, Gabriella Clare Marino⁠ and Amol Tyagi⁠.}

⁠Why NewsNest?

Checking a dozen news sites every day is slow, and most of what you see isn't for you. Algorithmic feeds fix that by learning what keeps you scrolling, which is not the same as what you want to read.

NewsNest is the opposite deal. You choose the sources, you write down what interests you, and you can see why every story ranks where it does. It runs on your own machine, keeps everything in one SQLite file, and talks to the Anthropic API with your own key.

⁠Features

• 📰 Your own sources. Any RSS or Atom feed, plus GitHub release feeds for software you run. Import an OPML file, or try new feeds on probation: they only earn a place in your feed once you read them.
• 🧠 Ranked for you. Claude scores each story on topic, importance and personal fit against an interests file that you write yourself or have drafted from a few ratings.
• 🧩 One story, not five. Coverage of the same event by several outlets is grouped into one card, and wide coverage lifts a story up.
• ✨ Key points on demand. A one-line lede, a few bullets, a "why this matters to you" line, and a hint whether the full article is worth your time.
• 👍 It learns from you. Likes, dislikes and opens shift what you see. Pick more about or less from for an outlet or topic, and review the weekly suggestions for your interests before they apply.
• 🔎 No black box. Calibration mode shows every story with its scores and reasons, and Missed this lets you paste a link the feed should have shown you.
• 🃏 Swipe to sort. A one-thumb deck to rate a backlog in minutes.
• 💸 A hard spending cap. AI calls stop for the day when your budget is reached (default $0.50). Without an API key everything still works, and stories are shown newest first.
• 📱 Installable and private. A mobile-first web app with light and dark themes that installs to your home screen. Password-protected, with a one-click database backup.
• 🩺 Knows when a feed breaks. A status page and a /health endpoint report sources that fail or go silent.

⁠Quick start

You need Docker⁠ with Compose, and, for the AI features, an Anthropic API key⁠. Pull the published image (no clone needed, and the one to use for a Portainer stack), or build from source.

⁠Option A: pull the published image

Multi-arch (linux/amd64 + linux/arm64, so a Raspberry Pi or an ARM NAS works too):

• GHCR: ghcr.io/lucifix/newsnest
• Docker Hub: docker.io/lucifix/newsnest

Paste this into Portainer (Stacks → Add stack → Web editor) or save it as docker-compose.yml anywhere. Only the password needs filling in:

services:
  newsnest:
    image: ghcr.io/lucifix/newsnest:latest
    container_name: newsnest
    restart: unless-stopped
    ports:
      - 3095:8080
    environment:
      NEWSNEST_PASSWORD: change-me   # required: with no password the app refuses all access
      ANTHROPIC_API_KEY: ""          # optional: enables ranking and summaries
      TZ: UTC                        # your time zone, e.g. Europe/Ljubljana
    volumes:
      - newsnest-config:/config
      - newsnest-data:/data

volumes:
  newsnest-config:
  newsnest-data:

For the full file, with host folders you can open and edit instead of Docker volumes, use docker-compose.yml⁠ from this repo:

mkdir newsnest && cd newsnest
curl -fsSLO https://raw.githubusercontent.com/Lucifix/newsnest/main/docker-compose.yml
curl -fsSL https://raw.githubusercontent.com/Lucifix/newsnest/main/.env.example -o .env
mkdir -p config data     # so Docker doesn't create them as root
# edit .env: set NEWSNEST_PASSWORD, and ANTHROPIC_API_KEY for AI
docker compose up -d

Update: docker compose pull && docker compose up -d. Set NEWSNEST_IMAGE_TAG in .env to pin a release such as 0.3.1; the default is latest.

⁠Option B: build from source

git clone https://github.com/Lucifix/newsnest.git && cd newsnest
cp .env.example .env     # set NEWSNEST_PASSWORD, and ANTHROPIC_API_KEY for AI
mkdir -p config data     # so Docker doesn't create them as root
docker compose up -d --build

Update: git pull && docker compose up -d --build

⁠First run

Open http://localhost:3095⁠, log in with your password, and tap Set up my profile:

1. Say where you live. Your country and city tell Claude whose news is "big", and your time zone is picked up from the browser.
2. Add local news. NewsNest offers the local pack for your country (Australia, Austria, Canada, Croatia, France, Germany, Italy, Slovenia, the United Kingdom and the United States so far), or add your own feeds under Local in Settings → Feeds.
3. Rate about 30 headlines, and Claude drafts your interests from them.

On first start NewsNest copies a starter config.yaml⁠ into config/, with international news and tech feeds and no local ones. Change the feeds in Settings → Feeds, or edit the file. It is re-read on every run, so there is nothing to restart.

⁠How it works

flowchart LR
    A["RSS / Atom feeds<br/>GitHub releases"] --> B["Fetch &<br/>de-duplicate"]
    B --> C["Group the same story<br/>across outlets"]
    C --> D["Claude scores<br/>topic · importance · fit"]
    D --> E["Rank<br/>score × freshness × what you like"]
    E --> F(["Your feed"])
    F -- "likes · opens · ratings" --> E

A 60-second loop fetches the sources that are due, scores new stories in batches, and writes key points for the ones above your threshold. Scoring uses a small, cheap model with prompt caching; the rarer jobs (drafting your profile, the weekly suggestions) use a stronger one. Both are configurable.

What leaves your server: headlines and snippets, and for key points the article text, go to the Anthropic API using your key. Your feed list, ratings and database stay on your machine, and NewsNest itself sends no telemetry.

⁠Configuration

Everything lives in the config/ folder, which is re-read on every run:

Edits made in the Settings pages are written back to config.yaml, comments included.

Sections. Every feed belongs to one section, and the sections are the tabs along the bottom of the feed. A new install starts with Local, World and Tech. In Settings → Feeds → Sections you can rename them, add your own (Sports, Science, Cinema…), reorder them, and remove one (its feeds and stories move to a section you pick). Local and World are built in: they can be renamed but not removed, and share the News tab. A section is a tab once it has a feed; at most four can be tabs, and any further section shows as a chip under For you. The list is the sections: key in config.yaml. A config.yaml from before sections were editable has none and keeps the old four (Tech, Local, World, Lifestyle) until you change them, so nothing needs migrating and the database is untouched.

⁠Development

python3 -m venv .venv && .venv/bin/pip install -r requirements.txt pytest
.venv/bin/python -m pytest                 # run the tests
NEWSNEST_PASSWORD=dev NEWSNEST_SCHEDULER=off .venv/bin/python -m newsnest   # the app on :8080

The first run copies config.example/ to ./config/ (git-ignored). Two helpers are useful when adding or debugging sources:

.venv/bin/python -m newsnest.spike --source <feed-id>    # fetch one source and print parsed samples
.venv/bin/python -m newsnest.runner --no-ai              # one fetch-and-store tick, without AI

There is also a dev container with its own project name, port 3096 and the local ./config and ./data folders:

docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build

newsnest/
  web.py            FastAPI app: pages, settings, login, JSON endpoints
  runner.py         the 60-second loop: fetch due sources, then the AI steps
  fetchers/         RSS/Atom and GitHub-releases fetchers
  pipeline.py       store items, de-duplicate by URL, record source health
  grouping.py       group the same story across outlets
  scoring.py        AI scoring and the final score
  summaries.py      key points, fetched from the article text
  feed.py           ranking and the home feed
  profile*.py       interests, drafting, and the weekly suggestions
  discovery.py      OPML import and sources on probation
  db.py             SQLite schema and migrations
  templates/        Jinja templates      static/  CSS, JS, fonts, icons
config.example/     starter config, copied to config/ on first start
tests/              pytest suite

Planned work and known limits are tracked in TODO.md⁠.

⁠Contributing

Bug reports, ideas and pull requests are welcome. Found a bug? Settings → About & report a bug in the app opens a new issue with your version filled in. Please open an issue first for anything big. Run python -m pytest and ruff check . before you send a PR (CI runs the linter on every PR), and add a test for what you change.

A local news pack for your country is an easy first PR: add a LocalPack to LOCAL_PACKS in newsnest/packs.py⁠ with 8 to 10 national outlets across the spectrum, each checked with newsnest.discovery.check_feed (https, working, posting in the last two weeks).

⁠License

MIT⁠, except the bundled fonts in newsnest/static/fonts/⁠: JetBrains Mono⁠ and Space Grotesk⁠ are licensed under the SIL Open Font License 1.1⁠; their license files sit next to the fonts.