seatgeek/docker-mirror repository overview

⁠docker-mirror

This project will copy public DockerHub, Quay or GCR repositories to a private registry.

• docker-mirror⁠
  • Install / Building⁠
  • Using⁠
    • Adding new mirror repository⁠
    • Updating / resync an existing repository⁠
    • Update all repositories⁠
  • Example config.yaml⁠
  • Environment Variables⁠

⁠Install / Building

• make sure you got Go 1.15 or newer
  • OSX: brew install go
• make sure you have CGO enabled
  • export CGO_ENABLED=1
• clone this repository to $HOME/src/github.com/seatgeek/docker-mirror
• change your working directory to $HOME/go/src/github.com/seatgeek/docker-mirror
• run go install to build and install the docker-mirror binary into your $HOME/go/bin/ directory
  • alternative: go build to build the binary and put it in the current working directory

⁠Using

Make sure that your local Docker agent is logged into to ECR.

• To login to ECR private registries:
  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin ACCOUNT_ID.dkr.REGION.amazonaws.com
• To login to ECR public registries:
  aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws/YOUR_ECR_PUBLIC_SUFFIX
  Note that the region must be us-east-1 for ECR public registry authentication.

See AWS ECR documentation⁠ for more details

docker-mirror will automatically create the ECR repository on demand, so you do not need to login and do any UI operations in the AWS Console.

docker-mirror will look for your AWS credentials in all the default locations (env, ~/.aws/ and so forth like normal AWS tools do)

⁠Configuration File

There are several configuration options you can use in your config.yaml below. Please see the config.yaml file in the repository for a full example.

• ignore_tag: This option sets tags that can be ignored on pulls. (i.e. ignore_tag: - "*-alpine")

• match_tag: This option sets the tags that you want to match on for pulls. (i.e. match_tag: - "3*")

• max_tag_age: This option sets the max tag age you wish to pull from. (i.e. max_tag_age: 4w)

• name: This option sets the name of your repository. (i.e. name: elasticsearch)

• host: This options sets where do you want to mirror repositories from. Accepted values include hub.docker.com, quay.io and gcr.io. If not set, images will be pulled from Docker Hub.

• private_registry: This option allows you to set a private Docker registry prefix for docker pulls. It will prefix any of your name: options with the private_registry name and a slash to allow you to customize where your images are being pulled through. This is particularly useful if you use a proxy to dockerhub. i.e. (private_registry: "private-registry-name")

⁠Adding new mirror repository

• add the new repository to the config.yaml file
  • TIP: omit the max_tag_age for the initial sync to mirror all historic tags (match_tag is fine to use in all cases)
• run PREFIX=${reopsitory_name} docker-mirror to trigger a sync for the specific new repository (you probably don't want to sync all the existing repositories)
• add the max_tag_age filter to the newly added repository so future syns won't cosider all historic tags

⁠Updating / resync an existing repository

• run PREFIX=${reopsitory_name} docker-mirror to trigger a sync for the specific repository
  • TIP: Consider if the tags you want to sync fits within the max_tag_age and other filters

⁠Update all repositories

• run docker-mirror and wait (for a while)

⁠Example config.yaml

---
cleanup: true # (optional) Clean the mirrored images (default: false)
target:
  # where to copy images to
  # Below is an example of the ECR private registry.
  # To mirror repositories to a ECR public registry, replace this value with public.ecr.aws/YOUR_ECR_PUBLIC_ALIAS
  registry: ACCOUNT_ID.dkr.REGION.amazonaws.com

  # (optional) prefix all repositories with this name
  # ACCOUNT_ID.dkr.REGION.amazonaws.com/hub/jippi/hashi-ui
  prefix: "hub/"

# what repositories to copy
repositories:
    # will automatically know it's a "library" repository in dockerhub
  - name: elasticsearch
    match_tag: # tags to match, can be specific or glob pattern
      - "5.6.8" # specific tag match
      - "6.*"   # glob patterns will match
    ignore_tag: # tags to never match on (even if its matched by `tag`)
      - "*-alpine" # support both glob or specific strings

  - name: yotpo/resec
    host: hub.docker.com # mirror the repository from Docker Hub
    max_tag_age: 8w # only import tags that are 8w or less old

  - name: jippi/hashi-ui
    max_tags: 10 # only copy the 10 latest tags
    match_tag:
      - "v*"
        
  - name: kubebuilder/kube-rbac-proxy
    host: gcr.io # mirror the repository from Google Container Registry 

  - name: jippi/go-metadataproxy # import all tags

⁠Environment Variables