rabbir/mitm-api repository overview

⁠mitm-api

A remotely-callable mitmproxy control plane: start/stop proxy instances on demand, collect HTTPS request samples, and manage CA certificates.

中文文档⁠

⁠Overview

mitm-api serves the prepare phase of a "prepare + lightweight replay" scraping workflow. A client points a browser through the managed proxy once; mitm-api intercepts the HTTPS traffic and records the exact header order, header values, and cookies as request templates for the scraper to reuse.

Browser → mitm-api proxy (HTTPS decrypted) → upstream proxy → target site
                 ↓ capture
           SQLite samples
                 ↓ GET /session/{id}
           Your scraper

⁠Quick Start

The image is published automatically to Docker Hub on every push to master: rabbir/mitm-api⁠

docker run -d --name mitm-api \
  -p 9000:9000 \
  -p 48000-48500:48000-48500 \
  -v /srv/mitm-api/data:/data \
  -e MITM_API_TOKEN=your-secret-token \
  -e PROXY_AUTH=true \
  rabbir/mitm-api:latest

Or with Docker Compose (copy .env.example to .env and fill in the values):

docker compose up -d

Build locally:

docker build -t rabbir/mitm-api:latest .

⁠Quick Test

A smoke-test script is bundled inside the container. Copy it out, set the three variables at the top, and run it. Requires jq⁠.

docker cp mitm-api:/test.sh ./test.sh && chmod +x test.sh
# edit API, TOKEN, UPSTREAM at the top of the file, then:
./test.sh

The script runs 8 steps: health check → create session → download CA → HTTP via proxy → HTTPS via proxy (with TLS issuer verification) → fetch samples → destroy session → verify destroyed.

⁠API

All endpoints require Authorization: Bearer <MITM_API_TOKEN> when MITM_API_TOKEN is set.

Interactive docs are available at /docs (Swagger UI), /redoc, and /openapi.json.

POST /session — Create a session and return proxy connection details.

{ "upstream": "1.2.3.4:8000" }

upstream is optional. Omit it for a direct connection (no upstream proxy).

Response (PROXY_AUTH=true includes credentials):

{
  "id": "3f2504e0-4f89-41d3-9a0c-0305e82c3301",
  "proxy": {
    "host": "100.x.x.x",
    "port": 48001,
    "username": "3f2504e0-...",
    "password": "a1b2c3d4..."
  },
  "expires_at": "2026-06-04T12:30:00Z",
  "status": "ready"
}

GET /sessions — List all non-destroyed sessions (metadata only, no credentials).

GET /session/{id} — Get full session detail including captured samples. Also resets expires_at to now + SESSION_DEFAULT_TTL. Call periodically as a heartbeat to keep the session alive.

DELETE /session/{id} — Destroy the session, stop the proxy, release the port.

GET /ca.pem — Download the CA public certificate (mitmproxy-ca-cert.pem). The private key is never exposed.

GET /healthz — Health check. Returns {"status": "ok"}.

⁠Environment Variables

Fixed constants (not configurable): control API on 0.0.0.0:9000, SQLite at /data/mitm-api.db.

⁠CA Certificate Lifecycle

On startup mitm-api inspects CA_DIR:

⁠Security

• Run on a private network. Expose ports 9000 and 48000–48500 only on a WireGuard/Tailscale interface.
• Enable both MITM_API_TOKEN and PROXY_AUTH in production. The control API can launch an arbitrary HTTPS-decrypting proxy — it is more sensitive than the proxy port itself.
• Credentials are held in memory only; never written to disk in plaintext.
• The CA private key (mitmproxy-ca.pem) is never exposed through the API.

⁠Project Structure

mitm-api/
├── Dockerfile
├── docker-compose.yml
├── pyproject.toml
├── test.sh
├── mitm_api/
│   ├── main.py           # FastAPI app + mitmproxy lifespan
│   ├── config.py         # Environment variable parsing
│   ├── api/
│   │   ├── session.py    # Session CRUD endpoints
│   │   └── ca.py         # CA download + healthz
│   ├── core/
│   │   ├── mode_manager.py  # Runtime mode add/remove, port allocation
│   │   ├── session.py       # SessionInfo dataclass
│   │   ├── ca.py            # Startup CA validation
│   │   ├── deps.py          # FastAPI dependency providers
│   │   └── reaper.py        # TTL reaper + retention purge
│   ├── addon/
│   │   └── capture.py    # mitmproxy addon: flow tagging, proxy auth, sample capture
│   ├── storage/
│   │   ├── db.py         # aiosqlite helpers (WAL, single write-lock)
│   │   └── models.py     # CREATE TABLE statements
│   └── security/
│       └── auth.py       # Bearer token FastAPI dependency
└── tests/