jlcox1970/forward-proxy repository overview

⁠Overview

forward-proxy is a lightweight, high-performance HTTP proxy designed specifically for Varnish-based CDN architectures, particularly for serving Deb/RPM repositories (APT, DNF, YUM, EPEL, Fedora Metalink, Rocky Linux mirrorlist, etc).

It is implemented in Go, optimized for extremely high concurrency, and runs in a minimal scratch container with no shell, no glibc, and no OS dependencies.

⁠Key Features

• Transparent HTTP/HTTPS upstream proxy

  • Uses the incoming Host header to route requests
  • Respects and forwards Range, If-Modified-Since, If-None-Match, etc.
  • Supports automatic HTTPS upgrade (honoring X-Forwarded-Proto)

• Custom DNS resolver

  • Avoid poisoned local DNS by querying upstream resolvers (UPSTREAM_DNS)
  • Defaults to Cloudflare + Google public resolvers

• Mirrorlist & Metalink Engine

  • Rule-based dynamic mirroring using a single YAML config

  • Supports:

    • Rocky Linux /mirrorlist
    • Fedora /metalink
    • Fedora/OpenH264 variants
    • EPEL mirrors

  • Programmable with templates and matching rules

• Prometheus Instrumentation

  • Request counters
  • Duration histograms
  • Bytes sent/received per host
  • Upstream error counters
  • In-flight request gauge
  • Optional remote-host metrics

• Streaming Mode (Zero Buffering)

  • Files (including large RPM/DEB packages) are streamed directly from the origin to Varnish
  • No disk usage, no request restart internally

• Health & Readiness Endpoints

  • /healthz and /readyz

• Works perfectly with Varnish

  • Passes through all relevant headers
  • Varnish can resume partial downloads via Range headers
  • No caching performed by the proxy

⁠Configuration

All configuration can be done using environment variables:

⁠Mirror Config (YAML)

A full dynamic mirror configuration engine is included.

Example:

tmirrors:
  - name: rocky-mirrorlist
    host: mirrors.rockylinux.org
    path_prefix: /mirrorlist
    base_url: http://myrocky-mirror.local
    repo_split_pattern: "^(?P<base>.*?)-(?P<version>[0-9]+)$"
    rules:
      - name: altarch
        when: { repo_contains: altarch }
        template: "{base_url}/pub/sig/{version}/altarch/{arch}/altarch-common"
    default_template: "{base_url}/pub/rocky/{version}/{base}/{arch}/os/"

⁠Performance

This proxy is designed for very high throughput:

• Go HTTP/2-capable Transport
• Long-keepalive persistent upstream connections
• Thousands of concurrent downloads
• Microsecond latency overhead

Tested successfully with:

• apt
• dnf
• yum
• libdnf
• curl
• Varnish bereq pass-through

⁠Minimal Scratch Image

Built using:

FROM scratch
COPY forward-proxy /forward-proxy
COPY /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
ENTRYPOINT ["/forward-proxy"]

Total image size is typically under 15 MB, including CA certs.