jlcox1970/package-server repository overview

⁠Repository Management

This application provides a server to manage DEB, RPM, Helm and Docker repositories. It generates GPG keys and signs DEB metadata, produces APT and YUM repository metadata, and acts as a caching proxy for upstream container registries. It is a single static Go binary with no runtime dependencies, so it runs anywhere regardless of host distribution — the DEB and RPM repositories it serves are independent of the OS it runs on.

⁠Features

• Creates all directories required
• Generates GPG keys and signs DEB metadata in-process
• Produces APT and YUM repository metadata natively — no external tooling required
• Acts as a pull-through cache for docker.io, ghcr.io, quay.io, nvcr.io, registry.k8s.io and others
• Resumable, digest-verified blob caching that survives client disconnects
• Chainable: edge instances can proxy back to a central instance
• Pure Go, no runtime dependencies, runs from a scratch container

⁠DockerHub

https://hub.docker.com/repository/docker/jlcox1970/package-server/general⁠

docker pull jlcox1970/package-server:tagname

⁠Prerequisites

None at runtime. The server is pure Go and invokes no external processes — there is no os/exec anywhere in the codebase. You do not need dpkg-dev, apt-utils, createrepo, createrepo-c or gnupg installed, on either the host or in the container.

Everything those tools used to provide is now handled in-process:

Building requires Go 1.24 or later. Because nothing is shelled out to, the binary runs on a FROM scratch image with no base OS.

Note that a scratch image ships no CA bundle. Upstream registry fetches work regardless because the mirror client skips certificate verification, but any HTTPS basic_api_backends you configure will fail with x509: certificate signed by unknown authority. Use a base image with ca-certificates, or mount a CA bundle, if you need those.

⁠Building

1. Clone the repository:

   git clone https://gitlab.com/jlcox70/repository-server.git
   

   Copy

2. Navigate to the project directory:

   cd repository-server
   

   Copy

3. Build the application:

   go build -o repo-manager
   

   Copy

4. Run the source code

   go run .
   

   Copy

5. (Optional) Install the binary globally:

   sudo mv repo-manager /usr/local/bin/
   

   Copy

⁠Usage

⁠Environment variables

Truthy values for the boolean variables are 1, true, yes, on (case-insensitive). Any other non-empty value is treated as false, so REGISTRY_ENABLE_PROXY=enabled silently disables the proxy and the config file is never consulted. Leaving a variable unset is safer than setting it wrongly.

⁠Health endpoints

/healthz, /readyz and /livez all return 200 unauthenticated and bypass the registry auth path. Suitable for Kubernetes probes.

First boot generates an RSA GPG key in-process, which can take a while on a CPU-limited container. Use a startupProbe with a generous failureThreshold — if key generation fails the process exits before it starts listening.

⁠Server Command-Line Options

• --origin: Specifies the APT::FTPArchive::Release::Origin value for the APT repository.

  • Default: InternalPackages
  • Example: --origin="MyRepo"

• --components: Specifies the APT::FTPArchive::Release::Components value.

  • Default: main
  • Example: --components="main contrib"

• --label: Sets the APT::FTPArchive::Release::Label value.

  • Default: Custom Packages
  • Example: --label="MyRepo Packages"

• --architectures: Defines the APT::FTPArchive::Release::Architectures value.

  • Default: i386 amd64 all source
  • Example: --architectures="amd64 arm64"

• --suite: Sets the APT::FTPArchive::Release::Suite value.

  • Default: stable
  • Example: --suite="testing"

• --email: Specifies the email address for GPG key generation.

  • Default: [email protected]
  • Example: --email="[email protected]"

• --key_name: Specifies the user name for the GPG key.

  • Default: Package Server
  • Example: --key_name="MyRepo Key"

⁠Examples

1. Run Server with defaults:

   ./repo-manager
   

   Copy

2. Generate APT repository configuration with custom values:

   ./repo-manager --origin="MyRepo" --components="main contrib" --label="MyRepo Packages" --architectures="amd64 arm64" --suite="stable" --email="[email protected]" --key_name="MyRepo GPG Key"
   

   Copy

3. Upload DEB to repository

curl -F "file=@k8s-diagnostics_1.0.1_amd64.deb" http://{package server}/upload

1. Upload RPM to repository

curl -F "[email protected]_64.rpm" http://{package server}/upload

1. Docker repository

docker push {package server}/{username}/{container}

1. Helm Charts

helm repo add my-repo http://{package server}/charts
helm push {chart file} my-repo

⁠Configuration

⁠Repository Metadata

• APT Repository: Package indexes, Release and InRelease files are generated and signed in Go.
• YUM Repository: repodata including primary.xml is generated in Go.

Both are produced entirely in-process; no apt-ftparchive or createrepo invocation is involved.

⁠File Storage

All files are kept in /var/www and should be backed up.

Cached upstream content lives under __mirror__/<host>/<repo>/ inside that tree. It is an internal implementation detail and is excluded from /v2/_catalog, so registry browsers such as the VS Code Docker extension show only your own repositories.

⁠GPG Key Generation

The tool generates a GPG key for signing packages. The key is created with default settings or customized based on command-line arguments.

⁠Extended Container Repository Configuration File

auth:
    mode: basic
    basic_htpasswd_file: ./.htpasswd
    basic_api_backends:
        - https://{rest server auth}
    basic_api_timeout: 3s
    basic_api_cache_ttl: 5m

  # (Optional) Explicitly list path policies, though not required for 'none'
    policies:
        - path: /v2/controlled/
          required: push
        - path: /v2/public/
          required: none

registry:
  enableProxy: true
  fallbackHosts:
    - ghcr.io
    - docker.io
    - quay.io
    - nvcr.io
    - registry.k8s.io
  debug: false

Note that keys under auth: are snake_case while keys under registry: are camelCase.

Unknown keys are ignored silently, so a misspelling produces no error — check the startup log to confirm what actually loaded.

⁠mode

Three modes exist:

*** the htpasswd file is used first and only moves to REST auth if the user is not listed in the file

⁠rest server auth

The rest server needs to take an auth request body with {"username": "[name]","password":"[password]"} and return a JWT.
The JWT is then used by your container tools for auth; the api backend must also validate the JWT.
Only a 200 response will allow the user to access the container repository.

basic_api_timeout bounds each backend call (default 3s) and basic_api_cache_ttl caches accepted credentials (default 5m).

⁠basic api backends

You can supply a list of api backends to use and they will be checked in order until a 200 is returned, or the list is exhausted at which time a 401 is sent to the client to deny access.

⁠policies

Policies are matched by path prefix, first match wins, so list more specific paths before general ones. If no policy matches, the path is public.

Any unrecognised value is treated conservatively and requires auth.

For a pull-through cache that should serve anonymously but require credentials to publish, use required: push.

⁠Enable Proxy

enableProxy allows you to proxy via the repository to upstream container registries. Also settable with REGISTRY_ENABLE_PROXY.

When enabled, a request resolves an upstream in this order:

1. REGISTRY_UPSTREAM_MIRROR, if set. Authoritative — nothing else is consulted. This is how proxy chaining works: an edge instance forwards to a central instance, which does the real upstream fetching.
2. The ns query parameter, if the client sent one. containerd and CRI-O send ?ns=quay.io when configured as a mirror via hosts.toml, which identifies the intended registry exactly.
3. fallbackHosts, tried in order until one returns the content.

Docker Engine and Podman do not send ns, so their requests always walk the fallback list. Order it with your most-used registries first: a miss costs a full round trip per host before moving on.

fallbackHosts behaviour:

⁠Tag freshness

Whether a cached manifest is served directly or revalidated upstream depends on the shape of the reference:

Case is folded, so V1.2.9 is immutable and Latest is mutable.

If upstream is unreachable or rate-limited, revalidation degrades gracefully and the cached manifest is served.

Note that floating version tags such as ubuntu:24.04 or node:20 begin with a digit and are therefore treated as immutable, even though upstream rebuilds them. Pull by digest, or purge the cache, if you need those refreshed.

⁠Blob caching

Layer transfers are designed to make forward progress under unreliable networks:

• Resumable. A partial download is kept and continued with a Range request on the next attempt rather than restarting from zero. Large layers converge over several attempts instead of looping.
• Digest verified. The full blob is hashed and checked against its digest before being promoted into the cache, so a truncated or corrupted transfer is never cached under a digest asserting it is valid.
• Idle timeout, not a deadline. A transfer is aborted only after 60 seconds with no bytes received. A slow but progressing multi-GB layer runs as long as it needs.
• Survives client disconnect. If a client hangs up mid-pull, the fetch continues to completion and is cached, so the next pull is a cache hit.

Interrupted transfers leave partial-<digest> files in the blob directory. These are removed by the cache purger, which requires REGISTRY_CACHE_PURGE_TTL or REGISTRY_CACHE_MAX_BYTES to be set.

⁠Logging

All requests to the server are logged, including POST requests with filenames and other details. Set REGISTRY_DEBUG=true for request headers and upstream proxy detail.

Useful log lines when diagnosing the proxy:

If a pull returns 404 with no mirror: lines and a near-zero duration, the proxy is disabled — check mirror_enabled above.

⁠License

This project is licensed under the MIT License. See the LICENSE⁠ file for more information.