rapid7/container-registry-sync-app repository overview

⚠️Use of this application requires a valid Rapid7 InsightVM license.

⁠Container Registry Sync App

Application for syncing Docker Registry, Repository, and Image data to Rapid7 InsightVM for vulnerability assessment. For more information view the help page⁠. For additional assistance, see Get Support⁠.

⁠Overview

This image collects repository and image data from a Docker Registry and uploads parsed data to Rapid7 InsightVM cloud for assessment. Raw images are not uploaded to Rapid7.

This image can be launched as either a long-running, persistent or semi-persistent service, or as on-demand one-shot runs. By default if left running it will sync the registry 1 hour after the previous sync completed, indefinitely.

⁠Requirements

• InsightVM license with cloud features enabled (registered to a specific region)
• A Rapid7 Insight API key that has Administrative access to InsightVM. See Managing Platform API Keys⁠.
• A Docker Registry that supports the Docker Registry HTTP API V2⁠

⁠Usage

Launch with environment variables:

docker run -e "REGISTRY_NAME=registry.internal.name" [... additional environment variables] rapid7/container-registry-sync-app:latest

Use -d to background the container execution. Use --rm to delete the container and all of its files upon exit.

Launch with mounted YAML configuration file:

docker run -v /path/to/config.yml:/data/config/application-prod.yml -e "MICRONAUT_CONFIG_FILES=/data/config/application-prod.yml" rapid7/container-registry-sync-app:latest

⁠Docker Compose

Example docker compose file. See Environment Variables section for additional information.

version: "3.7"
services:
  registry-sync-app:
    image: registry-sync-app:tag
    # (Optional) expose port 8080 for additional funcitonality
    ports:
      - "8080:8080"
    environment:
      # Insight API key (required)
      - INSIGHTVM_KEY=
      # One of: us, us2, us3, eu, ca, ap, au (required)
      - INSIGHTVM_REGION=
      # Short, descriptive registry name (required)
      - REGISTRY_NAME=
      # Full registry URL such as https://registry.internal:5000
      - REGISTRY_URL=
      - REGISTRY_USERNAME=
      - REGISTRY_PASSWORD=
      # Additional, optional environment variables
    volumes:
      # Optional file mounting (see next sections)
      - /path/to/cacerts:/opt/jre/lib/security/cacerts
      - /path/to/config.yml:/data/config/application-prod.yml

⁠TLS (HTTPS) support

This application can connect over HTTP or HTTPS to a Docker Registry. For HTTPS it expects a publicly trusted certificate (via OpenJDK cacerts) by default. However if you use an internal or self-signed certificate you can either mount a cacerts file which contains your certificate(s) or use the insecure flag to disable certificate validation (not recommended).

To add your certificate to the cacerts file, perform the following steps;

# copy cacerts file to current working directory
docker run --rm --entrypoint cat registry-sync-app:latest /opt/jre/lib/security/cacerts > cacerts

# use java keytool to add your certificate to the cacerts file
keytool -keystore cacerts -storepass changeit -noprompt -trustcacerts -importcert -alias registryroot -file registry.cer

#Launch with mounted cacerts file:
docker run -v /path/to/cacerts:/opt/jre/lib/security/cacerts [... env vars etc] rapid7/container-registry-sync-app:latest

⁠Built-in HTTP server

There is a built-in HTTP server on port 8080 which exposes additional functionality:

GET /info returns application version information

GET /health returns application health status: connectivity to your Docker Registry and InsightVM API, free disk space

POST /sync/all syncs all repositories and images, same as the scheduled task, but on-demand

POST /sync/repo/{repository} syncs a specific repository and its images. Useful if your Docker Registry doesn't support the /v2/_catalog endpoint or if it's otherwise unusable.

To enable access to the HTTP server, add -p 8080:8080 to your docker run command.

⁠Data Usage

This application will pull each image from the registry to perform analysis and upload metadata to the Rapid7 Insight Platform. Each image is composed of individual layer tar files which are downloaded compressed from the registry. Layer files can be re-used across images so they are currently not deleted to avoid re-downloading them. The files can be deleted manually if disk space becomes an issue.

⁠Parallel Sync

For very large registries this application can be deployed in a parallelized configuraiton to speed up the sync process. This means running multiple instances of the application configured to sync specific repositories. This strategy can also be used for registries that do not provide data via the /v2/_catalog endpoint or cannot paginate through large datasets.

To run the application for parallel sync, the schedule should be effectively disabled by setting the SYNC_INITDELAY value to 9999h. Assuming you have a list of repositories for your registry and you have divided them into separate lists based on the number of instances you will run, send a POST for each repository to the designated instance of the application.

For example, you can have a cron job loop over a file containing repository names which sends a curl command:

curl -X POST "http://container-registry-sync-app-instance:8080/sync/repo/${repository}"

If not using curl, be sure to URL Encode⁠ the repository names.

⁠Configuration

⁠API Key

A Rapid7 Insight API key is required for an account that has Administrative access to InsightVM. See Managing Platform API Keys⁠.

The API region must match the region where your InsightVM cloud instance is registered. Supported regions are: us, us2, us3, eu, ca, au, ap. See: Supported Regions⁠.

⁠Environment Variables

⁠YAML File

See Environment Variables section for explanation of each option.

Minimal configuration:

# minimal required configuration
insightvm:
  key: # insight API key
  region: # one of: us, us2, us3, eu, ca, au, ap
registry:
  name:
  # if your registry's /v2 API path is nested add the additonal path up to, but not including, /v2
  url: https://registry.internal:5000
  # credentials may be omitted if registry doesn't require any
  username: user
  password: pass

All options:

# full configuration options
insightvm:
  key:
  region:
  proxy:
    host: localhost
    port: 9000
    username: proxy
    password: proxy
registry:
  name:
  url: https://registry.internal:5000
  username: user
  password: pass
  insecure: false
  batchsize: 500
  dtr: false
  proxy:
    host: localhost
    port: 9000
    username: proxy
    password: proxy
sync:
  interval: 1h
  initdelay: 5s