Sign inSign up

drzippie/swaipy-converter

By drzippie

•Updated 3 months ago

Dockerized FastAPI service: PDF/DOCX/PPTX/HTML->Markdown (Docling/MarkItDown) & Markdown/HTML->PDF

Image
0

136

drzippie/swaipy-converter repository overview

⁠swaipy-converter

Dockerized document conversion service for swaipy.

  • PDF / DOCX / PPTX → Markdown using Docling⁠ (ML layout + OCR) — for feeding documents to a swaipy agent.
  • HTML / CSV / JSON / XML / EPUB / XLSX → Markdown using MarkItDown⁠ (Microsoft, lightweight).
  • Markdown / HTML → PDF using WeasyPrint⁠ (Markdown → HTML → PDF), plus Markdown → HTML.

Built with FastAPI. A single generic POST /convert endpoint drives every conversion. Internet-facing, so it is protected by Bearer-token authentication.

⁠Quick start (Docker)

# 1. Create your env file and set at least one API token
cp .env.example .env
python -c "import secrets; print('SWAIPY_API_TOKENS=' + secrets.token_urlsafe(32))" >> .env

# 2. Build and run
docker compose up --build

The first build downloads the Docling models (~1 GB) and bakes them into the image, so the container needs no network at runtime.

Service listens on http://localhost:8000. Interactive API docs at http://localhost:8000/docs.

⁠Endpoints

MethodPathAuthDescription
GET/health—Liveness probe.
GET/formats—List supported from → to conversions.
POST/convertBearerConvert a document (see below).
⁠POST /convert

Specify the input format (from), output format (to), the file, and optional options. Two request shapes, chosen by Content-Type:

  • multipart/form-data — binary file + form fields from, to, options (JSON string). Returns the converted result directly.
  • application/json — { "from", "to", "file_base64", "options" }. Returns a JSON envelope with the result inline (text) or base64 (binary).

Valid input formats: pdf, docx, pptx, html, csv, json, xml, epub, xlsx, markdown. Output formats: markdown, html, pdf. Query live support with GET /formats.

⁠Supported conversions
#FromToEngineOutputOptions
1pdfmarkdownDoclingtext (text/markdown)do_ocr, do_table_structure
2docxmarkdownDoclingtext (text/markdown)do_table_structure
3pptxmarkdownDoclingtext (text/markdown)do_table_structure
4pdfhtmlDoclingtext (text/html) — enriched, higher layout fidelity than Markdowndo_ocr, do_table_structure
5htmlmarkdownMarkItDowntext (text/markdown)—
6csvmarkdownMarkItDowntext (text/markdown)—
7jsonmarkdownMarkItDowntext (text/markdown)—
8xmlmarkdownMarkItDowntext (text/markdown)—
9epubmarkdownMarkItDowntext (text/markdown)—
10xlsxmarkdownMarkItDowntext (text/markdown)—
11markdownpdfpython-markdown → WeasyPrintbinary (application/pdf)lang, template
12htmlpdfWeasyPrintbinary (application/pdf)lang, template
13markdownhtmlpython-markdowntext (text/html)lang, template

Two engines: Docling (ML layout/OCR — best for PDF and Office docs) and MarkItDown (Microsoft, lightweight — for HTML and structured/text formats).

Format matrix (rows = input, columns = output):

↓ from \ to →pdfmarkdownhtml
pdf➖✅✅
docx—✅—
pptx—✅—
html✅✅➖
csv—✅—
json—✅—
xml—✅—
epub—✅—
xlsx—✅—
markdown✅➖✅

Templates for → pdf / → html: default, report (serif, book-like, page numbers), compact (denser). Select via options.template.

Full request/response reference: docs/API.md⁠.

⁠Examples

Set your token first:

TOKEN=your-token-here
BASE=http://localhost:8000
⁠curl — one per conversion (multipart)
# 1. PDF -> Markdown
curl -s -H "Authorization: Bearer $TOKEN" \
  -F "from=pdf" -F "to=markdown" \
  -F "[email protected]" \
  "$BASE/convert" -o output.md

# 1b. PDF -> Markdown, disabling OCR (faster for text-only PDFs)
curl -s -H "Authorization: Bearer $TOKEN" \
  -F "from=pdf" -F "to=markdown" \
  -F 'options={"do_ocr": false}' \
  -F "[email protected]" \
  "$BASE/convert" -o output.md

# 2. DOCX -> Markdown
curl -s -H "Authorization: Bearer $TOKEN" \
  -F "from=docx" -F "to=markdown" \
  -F "[email protected]" \
  "$BASE/convert" -o output.md

# 3. PPTX -> Markdown
curl -s -H "Authorization: Bearer $TOKEN" \
  -F "from=pptx" -F "to=markdown" \
  -F "[email protected]" \
  "$BASE/convert" -o output.md

# 4. PDF -> HTML (enriched, better layout fidelity than Markdown)
curl -s -H "Authorization: Bearer $TOKEN" \
  -F "from=pdf" -F "to=html" \
  -F "[email protected]" \
  "$BASE/convert" -o output.html

# 5. Markdown -> PDF (with language + template)
curl -s -H "Authorization: Bearer $TOKEN" \
  -F "from=markdown" -F "to=pdf" \
  -F 'options={"lang": "es", "template": "report"}' \
  -F "[email protected]" \
  "$BASE/convert" -o output.pdf

# 6. HTML -> PDF
curl -s -H "Authorization: Bearer $TOKEN" \
  -F "from=html" -F "to=pdf" \
  -F 'options={"template": "compact"}' \
  -F "[email protected]" \
  "$BASE/convert" -o output.pdf

# 7. Markdown -> HTML
curl -s -H "Authorization: Bearer $TOKEN" \
  -F "from=markdown" -F "to=html" \
  -F "[email protected]" \
  "$BASE/convert" -o output.html

# 8. HTML -> Markdown  (MarkItDown)
curl -s -H "Authorization: Bearer $TOKEN" \
  -F "from=html" -F "to=markdown" \
  -F "[email protected]" \
  "$BASE/convert" -o output.md

# 9. CSV / JSON / XML / EPUB / XLSX -> Markdown  (MarkItDown; just change 'from')
curl -s -H "Authorization: Bearer $TOKEN" \
  -F "from=csv" -F "to=markdown" \
  -F "[email protected]" \
  "$BASE/convert" -o output.md
⁠curl — JSON + base64
# Markdown -> PDF via JSON; decode the returned base64 into a PDF
curl -s -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
        \"from\": \"markdown\",
        \"to\": \"pdf\",
        \"file_base64\": \"$(base64 -i report.md)\",
        \"options\": { \"lang\": \"es\" }
      }" \
  "$BASE/convert" \
  | python -c "import sys,json,base64; open('output.pdf','wb').write(base64.b64decode(json.load(sys.stdin)['content_base64']))"
⁠Laravel / PHP

Using the built-in HTTP client (Illuminate\Support\Facades\Http). Configure the base URL and token in config/services.php / .env:

// .env
SWAIPY_CONVERTER_URL=http://converter:8000
SWAIPY_CONVERTER_TOKEN=your-token-here
// config/services.php
'swaipy_converter' => [
    'url'   => env('SWAIPY_CONVERTER_URL'),
    'token' => env('SWAIPY_CONVERTER_TOKEN'),
],

Multipart — PDF → Markdown (returns Markdown text):

use Illuminate\Support\Facades\Http;

$response = Http::withToken(config('services.swaipy_converter.token'))
    ->attach('file', file_get_contents($pdfPath), 'input.pdf')
    ->post(config('services.swaipy_converter.url').'/convert', [
        'from'    => 'pdf',
        'to'      => 'markdown',
        'options' => json_encode(['do_ocr' => true]),
    ]);

$response->throw();
$markdown = $response->body();

Multipart — Markdown → PDF (returns raw PDF bytes):

$response = Http::withToken(config('services.swaipy_converter.token'))
    ->attach('file', $markdownString, 'report.md')
    ->post(config('services.swaipy_converter.url').'/convert', [
        'from'    => 'markdown',
        'to'      => 'pdf',
        'options' => json_encode(['lang' => 'es']),
    ]);

$response->throw();

// Return as a download, or store it:
return response($response->body(), 200)
    ->header('Content-Type', 'application/pdf')
    ->header('Content-Disposition', 'attachment; filename="document.pdf"');

JSON + base64 — Markdown → PDF:

$response = Http::withToken(config('services.swaipy_converter.token'))
    ->acceptJson()
    ->post(config('services.swaipy_converter.url').'/convert', [
        'from'        => 'markdown',
        'to'          => 'pdf',
        'file_base64' => base64_encode($markdownString),
        'options'     => ['lang' => 'es'],
    ]);

$response->throw();
$pdfBytes = base64_decode($response->json('content_base64'));
Storage::put('documents/report.pdf', $pdfBytes);

Multipart — DOCX → Markdown (same pattern, just change from/to):

$response = Http::withToken(config('services.swaipy_converter.token'))
    ->attach('file', file_get_contents($docxPath), 'input.docx')
    ->post(config('services.swaipy_converter.url').'/convert', [
        'from' => 'docx',
        'to'   => 'markdown',
    ]);

$response->throw();
$markdown = $response->body();

The three shapes above (multipart→text, multipart→binary, JSON+base64) cover every conversion — just change from/to and the options. Text outputs (markdown, html) come back in the response body; binary outputs (pdf) come back as raw bytes (multipart) or content_base64 (JSON).

Tip: wrap these calls in a small SwaipyConverter service class and inject it where you need conversions, so the URL/token/error-handling live in one place.

⁠Local development

Requires uv⁠ and the WeasyPrint system libraries (Pango, cairo, GDK-PixBuf — see the Dockerfile for the full list; on macOS: brew install pango gdk-pixbuf libffi).

uv sync --extra dev
export SWAIPY_API_TOKENS=dev-token
uv run uvicorn app.main:app --reload
⁠Tests
uv run pytest

On macOS, if WeasyPrint can't find its native libs, prefix with DYLD_FALLBACK_LIBRARY_PATH=/opt/homebrew/lib.

The API and PDF-rendering tests run without downloading Docling models. The PDF→Markdown path is exercised via integration testing against a running container.

⁠Configuration

All settings use the SWAIPY_ env prefix (see app/config.py):

VariableDefaultDescription
SWAIPY_API_TOKENS(empty)Comma-separated Bearer tokens. Empty disables auth — never in production.
SWAIPY_MAX_UPLOAD_BYTES52428800 (50 MiB)Max upload size.
SWAIPY_DOCLING_DO_OCRtrueDefault: run OCR on scanned/image PDFs.
SWAIPY_DOCLING_DO_TABLE_STRUCTUREtrueDefault: recover table structure.
SWAIPY_CORS_ORIGINS*Comma-separated allowed CORS origins.

Per-request options on /convert override the OCR/table defaults.

⁠Architecture

app/
├── main.py                     FastAPI app, generic /convert dispatch, upload handling
├── auth.py                     Bearer-token dependency (constant-time compare)
├── config.py                   Env-driven settings
└── converters/
    ├── registry.py             (from, to) -> converter mapping + option models
    ├── docling_convert.py       Docling: PDF/DOCX/PPTX -> Markdown or enriched HTML
    ├── markitdown_convert.py    MarkItDown: HTML/CSV/JSON/XML/EPUB/XLSX -> Markdown
    ├── pdf_to_markdown.py        Back-compat shim over docling_convert
    ├── templates.py             Named CSS templates (default / report / compact)
    └── to_pdf.py                Markdown -> HTML -> PDF (WeasyPrint)

Conversions are CPU/memory-heavy and synchronous; the endpoint offloads them to a thread pool (run_in_threadpool) so the event loop stays responsive. Adding a new conversion means registering one entry in registry.py — no endpoint changes.

Tag summary

Content type

Image

Digest

sha256:96b997ccd…

Size

1 GB

Last updated

3 months ago

docker pull drzippie/swaipy-converter