bankja/marker repository overview

⁠Marker PDF - Docker Deployment Guide

This guide explains how to deploy Marker PDF using Docker with both GPU and CPU support.

⁠Table of Contents

• Running Container⁠
• Configuration⁠
• OpenWebUI Integration⁠
• Volume Mounts⁠
• Troubleshooting⁠

⁠Running Container

docker run -d \
  --name marker-pdf-gpu \
  --gpus all \
  -p 8001:8001 \
  -v marker-models:/root/.cache/huggingface \
  -v marker-torch:/root/.cache/torch \
  -v ./uploads:/app/uploads \
  -v ./output:/app/output \
  -e USE_LLM=true \
  -e OPENAI_API_KEY=your-api-key \
  marker-pdf:gpu

⁠Configuration

⁠Environment Variables

The following environment variables can be configured:

⁠Server Configuration

⁠LLM Service Configuration

Note: The first available LLM provider (based on configured API keys) will be auto-selected in this priority order:

1. GEMINI_API_KEY → GoogleGeminiService
2. OPENAI_API_KEY → OpenAIService
3. CLAUDE_API_KEY → ClaudeService
4. AZURE_API_KEY → AzureOpenAIService
5. OLLAMA_BASE_URL → OllamaService
6. Default → GoogleGeminiService (with placeholder key for local testing)

OpenWebUI Control: Use the use_llm toggle in OpenWebUI to enable/disable LLM processing per request.

⁠OpenAI Configuration

⁠Gemini Configuration

⁠Claude Configuration

⁠Azure OpenAI Configuration

⁠Ollama Configuration

⁠Processing Options (OpenWebUI Compatible)

⁠Advanced Options

⁠Example Configurations

⁠OpenAI with Custom Settings

environment:
  - USE_LLM=true
  - LLM_SERVICE=marker.services.openai.OpenAIService
  - OPENAI_API_KEY=sk-...
  - OPENAI_MODEL=gpt-4o
  - FORCE_OCR=true
  - OUTPUT_FORMAT=json
  - PAGINATE=true

⁠Ollama with Local Model

environment:
  - USE_LLM=true
  - LLM_SERVICE=marker.services.ollama.OllamaService
  - OLLAMA_BASE_URL=http://host.docker.internal:11434
  - OLLAMA_MODEL=llama3.2-vision
  - OUTPUT_FORMAT=markdown

⁠Claude with Azure

environment:
  - USE_LLM=true
  - LLM_SERVICE=marker.services.claude.ClaudeService
  - CLAUDE_API_KEY=sk-ant-...
  - CLAUDE_MODEL_NAME=claude-3-7-sonnet-20250219
  - MAX_CLAUDE_TOKENS=16384

⁠OpenWebUI Integration

Marker PDF is designed to work seamlessly with OpenWebUI. Here's how to configure it:

⁠How LLM Works with OpenWebUI

When integrated with OpenWebUI, the LLM feature works as follows:

1. Configure your LLM provider in environment variables (API keys in docker-compose.yml)
2. OpenWebUI sends use_llm: true/false in requests based on the toggle
3. Automatic provider detection: The first available API key determines which LLM provider is used
4. Per-request control: OpenWebUI toggle controls whether LLM is used for each document
5. Backward compatible: When use_llm is false, marker_server processes PDFs normally without LLM

Key Advantage: Set up your LLM provider once in environment variables - OpenWebUI handles the rest with its toggle.

⁠1. Add as a Tool in OpenWebUI

In OpenWebUI, add Marker PDF as an external tool with the following configuration:

Endpoint URL:

http://marker-pdf:8001/marker/upload

Method: POST

Content Type: multipart/form-data

⁠2. Available Toggles in OpenWebUI

• Use LLM: Enable LLM for better accuracy (requires LLM provider configured in environment)
• Skip Cache: Don't use cached results
• Force OCR: Force OCR on all pages
• Paginate: Add page numbers to output
• Strip Existing OCR: Remove existing OCR text
• Disable Image Extraction: Don't extract images
• Format Lines: Format output lines
• Output Format: Choose markdown/json/html

⁠3. Docker Compose Setup for OpenWebUI

services:
  marker-pdf:
    image: bankja/marker:cuda
    networks:
      - openwebui
    environment:
      - PORT=8001
      - HOST=0.0.0.0
      # Configure ONE LLM provider (first available will be used):
      # - OPENAI_API_KEY=${OPENAI_API_KEY}
      # - OPENAI_MODEL=gpt-4o-mini
      # OR
      - GEMINI_API_KEY=${GEMINI_API_KEY}
      - GEMINI_MODEL_NAME=gemini-2.0-flash
      # OR (uncomment for other providers)
      # - CLAUDE_API_KEY=${CLAUDE_API_KEY}
      # - OLLAMA_BASE_URL=http://host.docker.internal:11434
    volumes:
      - ./datalab:/root/.cache/datalab

  openwebui:
    image: ghcr.io/open-webui/open-webui:main
    ports:
      - "3000:8080"
    networks:
      - openwebui
    environment:
      - MARKER_API_URL=http://marker-pdf:8001
      # Your API keys for the LLM provider
      - OPENAI_API_KEY=${OPENAI_API_KEY}  # Or GEMINI_API_KEY, etc.

networks:
  openwebui:
    driver: bridge

⁠4. OpenWebUI Toggle Behavior

• When use_llm: true: Marker PDF will use the configured LLM provider (based on environment variables) for enhanced PDF processing
• When use_llm: false or not sent: Marker PDF processes PDFs normally without LLM, using original fast processing
• No API key configured: Falls back gracefully (may use placeholder key for local testing)

⁠Volume Mounts

⁠Required Volumes

Model Cache (Highly Recommended):

-v ./datalab:/root/.cache/datalab

These volumes persist downloaded models across container restarts. Without them, models (~2-3GB) will be re-downloaded each time.

⁠Optional Volumes

Uploads Directory:

-v ./uploads:/app/uploads

Persist uploaded files.

Output Directory:

-v ./output:/app/output

Access generated output files from the host.

Custom Configuration:

-v ./config:/app/config

Mount custom configuration files.

⁠Accessing the Server

Once the container is running:

• API Endpoint: http://localhost:8001⁠
• API Documentation: http://localhost:8001/docs⁠
• Health Check: http://localhost:8001/⁠

⁠Example API Request

# Upload and convert a PDF
curl -X POST "http://localhost:8001/marker/upload" \
  -F "[email protected]" \
  -F "output_format=markdown"

# Convert from filepath (if mounted)
curl -X POST "http://localhost:8001/marker" \
  -H "Content-Type: application/json" \
  -d '{
    "filepath": "/app/uploads/document.pdf",
    "output_format": "markdown",
    "use_llm": true
  }'

⁠Troubleshooting

⁠Models Keep Re-downloading

Problem: Models are downloaded every time the container starts.

Solution: Ensure you're using volume mounts for model cache:

-v ./datalab:/root/.cache/datalab

⁠GPU Not Detected

Problem: Container can't access GPU.

Solution:

1. Install NVIDIA Container Toolkit
2. Use --gpus all flag or docker-compose GPU configuration
3. Verify with: docker run --rm --gpus all marker-pdf:gpu nvidia-smi

⁠Out of Memory

Problem: Container runs out of VRAM/RAM.

Solution:

• GPU: Ensure at least 5GB VRAM available
• CPU: Reduce batch size with BATCH_MULTIPLIER=0.5
• Limit max pages: MAX_PAGES=100
• Enable VRAM-saving mode: Set FREE_VRAM_ON_IDLE=true to unload models after processing (uses ~0GB idle VRAM, adds ~2-5s per request for model loading)

⁠Low VRAM Devices (< 4GB)

Problem: Device has insufficient VRAM for standard operation (requires 5GB peak).

Solution: Enable dynamic VRAM management:

docker run -d \
  --name marker-pdf-gpu \
  --gpus all \
  -p 8001:8001 \
  -e FREE_VRAM_ON_IDLE=true \
  -v marker-models:/root/.cache/huggingface \
  -v marker-torch:/root/.cache/torch \
  marker-pdf:gpu

For systems using llama-swap with USE_LLM=true, additionally unload the llama-swap model before OCR:

docker run -d \
  --name marker-pdf-gpu \
  --gpus all \
  -p 8001:8001 \
  -e FREE_VRAM_ON_IDLE=true \
  -e USE_LLM=true \
  -e UNLOAD_LLAMA_SWAP_BASE_URL=http://172.17.0.1:9292 \
  -v marker-models:/root/.cache/huggingface \
  -v marker-torch:/root/.cache/torch \
  marker-pdf:gpu

Trade-offs:

• Pro: Idle VRAM usage drops significantly (~5GB → ~0GB), freeing up memory for other tasks.
• Con: Each request after an idle period is slower (~2-5s) due to model loading overhead.
• With UNLOAD_LLAMA_SWAP_BASE_URL: Additional VRAM freed during OCR phase (~1-2GB), allowing OCR to run on devices with very limited VRAM.

Performance Impact:

• With FREE_VRAM_ON_IDLE=false (default): Fast processing, high idle VRAM usage
• With FREE_VRAM_ON_IDLE=true: Slower first request per idle period, minimal idle VRAM
• With UNLOAD_LLAMA_SWAP_BASE_URL set: LLM model unloaded before OCR, maximizing available VRAM for OCR

⁠Port Already in Use

Problem: Port 8001 is already in use.

Solution: Change the port mapping:

-p 8002:8001  # Use port 8002 on host

Or set PORT environment variable:

-e PORT=8002 -p 8002:8002

⁠Connection Refused in OpenWebUI

Problem: OpenWebUI can't connect to Marker PDF.

Solution:

1. Ensure both containers are on the same network
2. Use container name as hostname: http://marker-pdf:8001
3. Check firewall rules
4. Verify with: docker compose logs marker-gpu

⁠Advanced Usage

⁠Multi-GPU Setup

deploy:
  resources:
    reservations:
      devices:
        - driver: nvidia
          count: 2  # Use 2 GPUs
          capabilities: [gpu]

⁠Support

For full details:

• Github: https://github.com/bankjaneo/marker⁠
• Marker Documentation: https://github.com/datalab-to/marker/blob/main/README.md⁠

⁠License

This Docker setup follows the same license as the Marker project.