cisagov/code-gov-update repository overview

⁠code-gov-update

⁠Docker Image

This project contains code for updating the DHS code.gov⁠ inventory file⁠.

⁠How it works

The LLNL/scraper⁠ project is used to scrape a handful of GitHub organizations that belong to DHS and produce an updated JSON file per the code.gov specification⁠. If that file differs from the previously-generated one, it is emailed to the appropriate address so that it can be used to update the content hosted by DHS⁠.

⁠Running

⁠Running with Docker

To run the cisagov/code-gov-update image via Docker:

docker run cisagov/code-gov-update:0.3.0-rc.9

⁠Running with Docker Compose

1. Create a compose.yml file similar to the one below to use Docker Compose⁠.

   ---
   name: code-gov-update
   
   services:
     update:
       environment:
         - AWS_CONFIG_FILE=path/to/aws_config
         - AWS_PROFILE=default
       image: cisagov/code-gov-update:0.3.0-rc.9
       init: true
   

   Copy

2. Start the container and detach:

   docker compose up --detach
   

   Copy

⁠Using secrets with your container

This container also supports passing sensitive values via Docker secrets⁠. Passing sensitive values like your credentials can be more secure using secrets than using environment variables. See the secrets⁠ section below for a table of all supported secret files.

1. To use secrets, create aws_config and scraper.json files containing the values you want set:

   [default]
   credential_source = Ec2InstanceMetadata
   region = us-east-2
   role_arn = arn:aws:iam::123456789012:role/AssumeSesSendEmail-CodeGovUpdate
   

   Copy

   Please see the documentation⁠ for creating your own scraper.json configuration file.

2. Then add the secrets to your compose.yml file:

   ---
   name: code-gov-update
   
   secrets:
     aws_config:
       file: ./src/secrets/aws_config
     scraper_config:
       file: ./src/secrets/scraper.json
   
   services:
     update:
       environment:
         - AWS_CONFIG_FILE=/run/secrets/aws_config
         - AWS_PROFILE=default
       image: cisagov/code-gov-update:0.3.0-rc.9
       init: true
       secrets:
         - source: aws_config
           target: aws_config
         - source: scraper_config
           target: scraper_config.json
   

   Copy

⁠Updating your container

⁠Docker Compose

1. Pull the new image from Docker Hub:

   docker compose pull
   

   Copy

2. Recreate the running container by following the previous instructions⁠:

   docker compose up --detach
   

   Copy

⁠Docker

1. Stop the running container:

   docker stop <container_id>
   

   Copy

2. Pull the new image:

   docker pull cisagov/code-gov-update:0.3.0-rc.9
   

   Copy

3. Recreate and run the container by following the previous instructions⁠.

⁠Updating Python dependencies

This image uses Pipenv⁠ to manage Python dependencies using a Pipfile⁠. Both updating dependencies and changing the Pipenv⁠ configuration in src/Pipfile will result in a modified src/Pipfile.lock file that should be committed to the repository.

⁠Updating dependencies

If you want to update existing dependencies you would run the following command in the src/ subdirectory:

pipenv lock

⁠Modifying dependencies

If you want to add or remove dependencies you would update the src/Pipfile file and then update dependencies as you would above.

⁠Image tags

The images of this container are tagged with semantic versions⁠ of the underlying example project that they containerize. It is recommended that most users use a version tag (e.g. :0.3.0-rc.9).

See the tags tab⁠ on Docker Hub for a list of all the supported tags.

⁠Volumes

There are no volumes.

⁠Ports

No ports are exposed by this container.

⁠Environment variables

⁠Required

There are no required environment variables.

⁠Optional

There are no optional environment variables.

⁠Secrets

⁠Building from source

Build the image locally using this git repository as the build context⁠:

docker build \
  --tag cisagov/code-gov-update:0.3.0-rc.9 \
  https://github.com/cisagov/code-gov-update.git#develop

⁠Cross-platform builds

To create images that are compatible with other platforms, you can use the buildx⁠ feature of Docker:

1. Copy the project to your machine using the Code button above or the command line:

   git clone https://github.com/cisagov/code-gov-update.git
   cd code-gov-update
   

   Copy

2. Create the Dockerfile-x file with buildx platform support:

   ./buildx-dockerfile.sh
   

   Copy

3. Build the image using buildx:

   docker buildx build \
     --file Dockerfile-x \
     --platform linux/amd64 \
     --output type=docker \
     --tag cisagov/code-gov-update:0.3.0-rc.9 .
   

   Copy

⁠Contributing

We welcome contributions! Please see CONTRIBUTING.md⁠ for details.

⁠License

This project is in the worldwide public domain⁠.

This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication⁠.

All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.