frugan/convsync repository overview

⁠ConvSync

Keeps a folder of documents mirrored as Markdown, so that LLM agents and RAG tooling can read them. Scanned documents get a text layer through local OCR first. Runs incrementally, cleans up after deleted documents and never overwrites a Markdown file you edited by hand. See the supported formats⁠.

No machine learning runtime inside — no numpy, no onnxruntime, no torch — so it runs on x86-64 CPUs without AVX2, where markitdown, docling and most of the AI-oriented converters abort with Illegal instruction (SIGILL). It also keeps the image small, the output deterministic, and the documents on your machine.

documents/                         documents/_md/
├── 2026/                          ├── 2026/
│   ├── invoice-4417.pdf     ──►   │   ├── invoice-4417.pdf.md
│   ├── scan-receipt.pdf     ──►   │   ├── scan-receipt.pdf.md   (via _ocr/)
│   └── assets.xlsx          ──►   │   └── assets.xlsx.md

⁠What is in the image

Tesseract selects its SIMD code path at runtime and falls back to SSE on CPUs without AVX2. Every CI build fails if numpy, onnxruntime, torch or magika end up installed.

⁠Supported formats

Every other file is ignored. Plain text formats (CSV, TXT, JSON, HTML) need no conversion: agents read them as they are. More formats are tracked in #5⁠.

⁠Available tags

Platforms: linux/amd64, linux/arm64.

⁠Quick start

docker run --rm --network none \
  -v "$PWD/documents:/data" \
  frugan/convsync:1

The Markdown lands in documents/_md/, the OCR copies of scanned PDFs in documents/_ocr/. Run it again whenever documents change: only what is new or modified is processed.

--network none is not required, but nothing in the pipeline needs the network: with it, the documents provably stay on the machine.

⁠File ownership

The container runs as root by default. With a Docker daemon configured for userns-remap, container uid 0 maps to your host user and the generated files are yours. Without it, pass your ids:

docker run --rm --network none --user "$(id -u):$(id -g)" \
  -v "$PWD/documents:/data" frugan/convsync:1

⁠Inside a Compose stack

The typical setup: the documents live in a gitignored folder of the project, and the agents get only the Markdown.

services:
  convsync:
    image: frugan/convsync:1
    profiles: ["tools"]        # never started by `docker compose up`
    network_mode: none
    volumes:
      - ./agents/data:/data

docker compose run --rm convsync

Mount only _md/ into the containers the agents run in, read-only (./agents/data/_md, with :ro): the originals and the OCR copies in _ocr/ then stay out of their reach. An agent running directly on the host can read the whole folder regardless.

⁠Commands

convsync [sync] [--force]   # OCR the scanned PDFs, then convert everything
convsync ocr [--force]      # only add the text layer to scanned PDFs
convsync convert [--force]  # only convert to Markdown
convsync bash               # anything else is executed as is

--force rebuilds everything instead of what changed. With sync it applies to both steps, which means re-OCRing every scan: use convert --force when only the Markdown has to be regenerated.

Every run ends with a summary, and the exit code is non-zero if any document failed:

ocr: 1 | skipped: 12 | native: 30 | removed: 0 | failed: 0
converted: 2 | skipped: 41 | removed: 1 | conflicts: 0 | failed: 0

⁠How it works

OCR only where it is needed. A PDF that yields fewer than 100 characters of text is treated as an image-only scan and goes through ocrmypdf into _ocr/<path>.ocr.pdf. So is a PDF whose every page is a full-page image, such as a filled-in form flattened back to PDF, whose text layer holds only the field values. Born digital PDFs are left alone: rasterizing them would only degrade the text. Originals are never modified.

Mirrored output. _md/ reproduces the source tree, keeping the original extension in the name, so report.pdf and report.xlsx in the same folder do not collide.

Incremental. A document is converted again only when it (or its OCR copy) is newer than its Markdown.

Orphans. When a document is deleted, its Markdown and its OCR copy go with it, and so do the folders left empty.

Hand edits are safe. Each generated file starts with a provenance stamp carrying the SHA-256 of its body:

<!-- generated from 2026/invoice-4417.pdf sha256:3f1a… -->

If the body no longer matches the stamp, someone edited it: the rebuild is written next to it as <name>.md.new and reported as a CONFLICT, instead of overwriting the edit. Merge by hand and delete the .new file; it also goes away on its own as soon as the edit and the converter agree again.

Output shape. PDF text is wrapped in a fenced code block, because the alignment produced by pdftotext -layout is meaningful on forms, invoices and tax documents, and Markdown would collapse it. XLSX sheets become one table per sheet, with the values cached by Excel or LibreOffice rather than the formulas.

⁠Configuration

Language packs are chosen at build time: the published image carries English and Italian. For other languages, build your own:

docker build --build-arg TESSERACT_LANGS="deu fra" -t convsync:custom .
docker run --rm -e OCR_LANGS=deu+fra+eng -v "$PWD/documents:/data" convsync:custom

⁠Comparison

Most converters are libraries that turn one file into Markdown, often with far better layout reconstruction than this image. What they do not do is keep a folder in sync, and the AI-based ones carry a runtime that rules out older CPUs. The table reflects the state of each project at the time of writing: check upstream before relying on it.

When to pick something else:

• Complex layouts (multi-column papers, formulas, nested tables) on a modern CPU: Docling, Marker or MinerU reconstruct the reading order far better than pdftotext.
• DOCX, PPTX, ODT, EPUB: Anydoc or MarkItDown. This image does not convert them (yet).
• One-off conversions: any library above; the sync machinery here only pays off on a folder that keeps changing.

⁠Local build

The build: section of docker-compose.yml is active. image is both the pull reference and the build tag, so set it in both commands: building with IMAGE unset overwrites frugan/convsync:latest in the local cache.

cp .env.dist .env
IMAGE=convsync:local docker compose build
IMAGE=convsync:local docker compose run --rm convsync

The documents go in ./data, which is gitignored. The end-to-end test needs no documents of yours:

docker run --rm --network none --entrypoint /opt/convsync/tests/smoke.sh convsync:local

⁠Limitations

• Only PDF and XLSX are converted; other files are ignored.
• A PDF that mixes born digital and scanned pages (a typed cover on a scanned body) clears the 100-character threshold and is not OCRed: only PDFs scanned on every page are caught regardless of their text.
• XLSX formulas never computed by a spreadsheet application (files written by scripts) have no cached value and come out empty.
• OCR quality is Tesseract's: fine on clean scans, poor on photos and handwriting.

⁠Contributing

For your contributions please use:

• Conventional Commits⁠
• Pull request workflow⁠

See CONTRIBUTING⁠ for detailed guidelines.

⁠Sponsor

⁠License

Copyright (c) 2026 Frugan⁠. MIT⁠, see the LICENSE⁠ file.

The bundled tools keep their own licenses: notably ocrmypdf (MPL-2.0), Tesseract (Apache-2.0), Ghostscript (AGPL-3.0) and poppler (GPL).