sonatype/gitlab-nexus-iq-pipeline

By sonatype

Updated 22 days ago

Image

1M+

Sonatype for GitLab CI

Sonatype for GitLab CI is packaged as a Docker image that allows you to perform policy evaluations against one or more build artifacts during a GitLab CI/CD pipeline run. You can also scan Docker containers using the Sonatype Container Security integration.

For GitLab Ultimate customers, Sonatype for GitLab CI can populate the Vulnerability Report and the Dependency List, available under the GitLab Ultimate Security feature.

Pipeline Environment Variables

We recommend setting up the following environment variables at the group or project level via <Project/Group>SettingsCI/CDVariables:

  • NEXUS_IQ_URL - Sonatype IQ Server URL
  • NEXUS_IQ_USERNAME - Sonatype IQ Server user name
  • NEXUS_IQ_PASSWORD - Sonatype IQ Server password (make sure to mask the password)
  • NEXUS_IQ_REPORT_FORMAT- Policy evaluation report format. Accepted values: summary and enhanced.

Specific to container scanning, for connecting to remote registries, the following variables must be defined as well. These are the credentials for your Docker registry:

  • NEXUS_CONTAINER_IMAGE_REGISTRY_USER - Docker registry user name
  • NEXUS_CONTAINER_IMAGE_REGISTRY_PASSWORD - Docker registry password

Sonatype IQ Policy Evaluation

Available for all GitLab tiers

The /sonatype/evaluate action allows you to perform policy evaluations against one or more build artifacts as part of a GitLab CI/CD pipeline.

Usage
myJob:
  stage: <GitLab pipeline stage>
  image: sonatype/gitlab-nexus-iq-pipeline:latest  
  script:
    /sonatype/evaluate [options] <Archives or directories to scan>

Example

policyEval:
  stage: test
  image: sonatype/gitlab-nexus-iq-pipeline:latest
  script:
    /sonatype/evaluate -i SomeWebApp target/our-web-app.war 
Available Options

-s, --server-url <iq-server-url>

IQ Server URL. If not provided, the NEXUS_IQ_URL environment variable must be set. Required

-a, --authentication <username:password>

IQ Server credentials. If not provided, the NEXUS_IQ_USERNAME and NEXUS_IQ_PASSWORD environment variables must be set. Required

-i, --application-id <iq-application-id>

The ID of the application on the IQ Server. Required

-t, --stage <stage>

The IQ Server stage to run analysis against. Accepted values: develop, source, build, stage-release, release, operate. Default: build

-O, --organization-id <iq-organization-id>

The ID of the organization on the IQ Server. This determines the organization under which the application will be created in case the application doesn't exist and the automatic application creation configuration is enabled. Default: none

-f, --report-format <format>

Controls the verbosity of policy evaluation HTML reports. Accepted values: summary and enhanced. If not provided and the NEXUS_IQ_REPORT_FORMAT is set, the environment variable value is used. Default: enhanced

-rn, --report-name <policy-evaluation-report.html>

Name of the policy evaluation HTML report. Default: {application-name}-policy-eval.html

-r, --result-file <result-file.json>

Name of the JSON file where the results of the policy evaluation will be stored in a machine-readable format. Default: none

-w, --fail-on-policy-warnings

Fail on policy evaluation warnings. Default: false

-e, --ignore-system-errors

Ignore system errors (IO, network, server, etc.). Default: false

-E, --ignore-scanning-errors

Ignore scanning errors (Corrupt files or malformed files, etc). Default: false

-p, --proxy <host[:port]>

Proxy to use; format host[:port]. Default: none

-U, --proxy-user <username:password>

Credentials for the proxy, format username:password. Default: none

-c, --callflow-analysis

Enable callflow analysis. Default: false

-cn, --callflow-analysis-namespaces

Runs callflow analysis only for the given namespaces. Can be specified more than once, e.g: -cn namespace1 -cn namespace2. Optional

-X, --debug

Enable debug logs. WARNING: This may expose sensitive information in the logs. Default: false

-h, --help

Shows the help screen.

-D<key=value>

Sets Java system properties as key-value pairs. Can be specified more than once, e.g: -Dkey1=value1 -Dkey2=value2. Optional

Evaluation Report

The evaluate action produces an HTML report that contains either:

  • brief summary information and a link to the detailed report on the IQ server, or
  • all the above plus details about the components that trigger policy actions, including policy violation details.

The evaluation report is stored in the build directory and is named -policy-eval-report.html by default. You can change the name using --report-name.

If you want to save the report as a pipeline artifact, you can add it in the artifacts section of the evaluation step in your .gitlab-ci.yml file, as follows:

  artifacts:
    paths:
      - <application-id>-policy-eval-report.html
Additional Examples

Multiple build artifacts, same IQ application ID

In this case you can specify multiple scan targets (directories or files), separated by spaces, as part of the same evaluation:

policyEval:
  stage: test
  image: sonatype/gitlab-nexus-iq-pipeline:latest 
  script:
    /sonatype/evaluate -i SomeSystem web-module/target/our-web-app.war lib-module/target/our-shared-lib.jar 

Multiple build artifacts, different IQ application IDs

In this case you can perform separate evaluations against each artifact. You can either do that within the same job or define separate jobs for each IQ application.

This example uses the same job:

policyEval:
  stage: test
  image: sonatype/gitlab-nexus-iq-pipeline:latest 
  script:
    /sonatype/evaluate -i SomeWebApp web-module/target/our-web-app.war
    /sonatype/evaluate -i SomeSharedLib lib-module/target/our-shared-lib.jar 

This example uses multiple jobs:

webPolicyEval:
  stage: test
  image: sonatype/gitlab-nexus-iq-pipeline:latest 
  script:
    /sonatype/evaluate -i SomeWebApp web-module/target/our-web-app.war
    
libPolicyEval:
  stage: test
  image: sonatype/gitlab-nexus-iq-pipeline:latest
  script:
    /sonatype/evaluate -i SomeSharedLib lib-module/target/our-shared-lib.jar 

Advanced pipeline (multiple stages with multiple policy evaluations)

This is a more complete example that illustrates one way of setting up a more realistic or typical pipeline with multiple stages and multiple policy evaluations.

A couple of important points:

  • It is recommended that each policy evaluation in the pipeline be configured to run at the appropriate IQ stage, as will be shown below.
  • It is important that policy evaluations for artifacts built from development/feature branches be run at the 'develop' IQ stage so as to not interfere with or create churn for the policy evaluation history of the default/main branch.

In this example we're going to use three pipeline stages for a maven project: build, test and release with a Nexus IQ policy evaluation for test and release. Since we're doing multiple evaluations we're going to setup a hidden template job that will consume an IQ_STAGE variable, as so:

.nexus_iq_policy_eval: &nexus_iq_policy_eval
  image: sonatype/gitlab-nexus-iq-pipeline:latest
  script:
    - /sonatype/evaluate -i SomeWebApp -t $IQ_STAGE target/*.war
  artifacts:
    paths:
      - ./SomeWebApp-policy-eval-report.html
      
  • The period before the job name marks the job as hidden so it will not show up or be executed directly as part of the pipeline.
  • The ampersand-prefixed name after the job name is an alias for the job name that can be referenced in other job definitions.

Next, we'll setup the three policy evaluations using the template created above.

There are two policy evaluations configured at the test stage, one for the default/main branch and one for all other branches.

The GitLab pipeline rules 'only' and 'except' are used to specify which job runs against which branch.

iq_policy_main:
  <<: *nexus_iq_policy_eval
  stage: test
  variables:
    IQ_STAGE : "build"
  only:
    - main

iq_policy_branch:
  <<: *nexus_iq_policy_eval
  stage: test
  variables:
    IQ_STAGE : "develop"
  except:
    - main

Those two jobs are mutually exclusive based on the name of the branch supplied to the pipeline rules.

  • Note: if your default branch is named something other than 'main' simply replace 'main' with the actual name of your default branch. Consult the GitLab documentation for more information.
  • Also, newer versions of GitLab, starting around version 12.3, are phasing in a new way of defining pipeline rules and may at some point deprecate the 'only' and 'except' rules shown above. So, pay particular attention to the documentation for the version you are using to know what's supported.

The release stage policy evaluation below uses the GitLab rule 'when' to specify that the given job will be manually started by a user. If all of the stages prior to release, in this example, have completed successfully GitLab will add a play button to the pipeline visualization that can be used to manually start the release policy evaluation job.

iq_policy_release:
  <<: *nexus_iq_policy_eval
  stage: release
  variables:
    IQ_STAGE : "release"
  only:
    - main
  when: manual

Here's a complete example showing all the jobs in proper context:

image: maven:latest

variables:
  MAVEN_CLI_OPTS: "--batch-mode"
  MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"

cache:
  paths:
    - .m2/repository/
    - target/

stages:
  - build
  - test
  - release

build:
  stage: build
  script:
    - mvn $MAVEN_CLI_OPTS clean install
  artifacts:
    name: "$CI_JOB_NAME-$CI_COMMIT_REF_NAME"
    when: on_success
    paths:
      - target/*.war

.nexus_iq_policy_eval: &nexus_iq_policy_eval
  image: sonatype/gitlab-nexus-iq-pipeline:latest
  script:
    - /sonatype/evaluate -i SomeWebApp -t $IQ_STAGE target/*.war
  artifacts:
    paths:
      - ./SomeWebApp-policy-eval-report.html
      
iq_policy_main:
  <<: *nexus_iq_policy_eval
  stage: test
  variables:
    IQ_STAGE : "build"
  only:
    - main

iq_policy_branch:
  <<: *nexus_iq_policy_eval
  stage: test
  variables:
    IQ_STAGE : "develop"
  except:
    - main

iq_policy_release:
  <<: *nexus_iq_policy_eval
  stage: release
  variables:
    IQ_STAGE : "release"
  only:
    - main
  when: manual
Container Scanning via Sonatype Container Security

In order to scan containers in GitLab CI/CD, the Docker in Docker image must be used. Further details on Docker in Docker can be found on the GitLab Docs.

The environmental variables defined in the settings must be passed to the script since they are needed in the docker container that will run within the docker they were passed in to. Along with explicitly passing in the parameters, an additional runtime parameter is needed:

-v /var/run/docker.sock:/var/run/docker.sock

A starter example can be seen below where webgoat-8.0 is scanned as a part of the build:

image: docker:docker:24.0.5

services:
  - docker:24.0.5-dind

build:
  stage: build
  script:
    - docker run -v $CI_PROJECT_DIR:/sonatype/reports -v /var/run/docker.sock:/var/run/docker.sock -e NEXUS_IQ_URL -e NEXUS_IQ_USERNAME -e NEXUS_IQ_PASSWORD -e NEXUS_CONTAINER_IMAGE_REGISTRY_USER -e NEXUS_CONTAINER_IMAGE_REGISTRY_PASSWORD sonatype/gitlab-nexus-iq-pipeline:latest /sonatype/evaluate -i SomeContainerApp container:https://registry.hub.docker.com/webgoat/webgoat-8.0
  artifacts:
    paths:
      - $CI_PROJECT_DIR/$CI_PROJECT_NAME-policy-eval-report.html

In the example above, NEXUS_CONTAINER_IMAGE_REGISTRY_USER and NEXUS_CONTAINER_IMAGE_REGISTRY_PASSWORD are only needed if the image being scanned is private. If the image being scanned is publicly accessible, credentials do not need to be passed in.

For versions 1.183.0 and earlier, a shared mount path must also be provided following the docker run call.

-v /tmp:/tmp

Vulnerability Report Update

Available for GitLab Ultimate

The /sonatype/create-vulnerability-report action can populate the Vulnerability Report under Secure section in GitLab based on data provided by a previous policy evaluation step.

This action reads an evaluate action generated result file and generates a dependency scanning report, which is automatically ingested by GitLab. To connect the two steps, the --result-file parameter must be used in the evaluate step. The result file must also be exposed as a pipeline artifact in the evaluate step, in this way it is made available to other steps.

GitLab will ingest the generated vulnerability report only if it is set as a pipeline artifact of type: dependency_scanning.

Note: This step will connect to the IQ Server to retrieve vulnerability details. The credentials set for the evaluate step will be used.

Usage
dependency_scanning:
  needs: [ <evaluate-step-name> ]
  stage: <GitLab pipeline stage>
  when: always
  image: sonatype/gitlab-nexus-iq-pipeline:latest
  script:
    - /sonatype/create-vulnerability-report [options]
  artifacts:
    reports:
      dependency_scanning: <vulnerability-report.json>

Example

iq_policy_evaluation:
  stage: test
  image: sonatype/gitlab-nexus-iq-pipeline:latest
  script:
    - /sonatype/evaluate -i testapp -r scan-result.json *.jar
  artifacts:
    when: always
    paths:
      - scan-result.json

dependency_scanning:
  needs: ["iq_policy_evaluation"]
  stage: test
  when: always
  image: sonatype/gitlab-nexus-iq-pipeline:latest
  script:
    - /sonatype/create-vulnerability-report -r scan-result.json --report-file vulnerability-report.json
  artifacts:
    reports:
      dependency_scanning: vulnerability-report.json 
Available Options

-r, --result-file <result-file.json>

The name of the JSON file where the results of the policy evaluation were stored. Required

-f, --report-file <vulnerability-report.json>

The name of generated dependency scanning report, which is used to populate the Vulnerability Report section in GitLab. Required

Fetch and Store SBOM Files

Available for all GitLab tiers

The /sonatype/fetch-sbom action can download an SBOM file associated with a previous policy evaluation step from Sonatype IQ Server, more precisely an SBOM file associated with a specific scan ID. The SBOM file is also stored as a pipeline artifact.

The evaluate stores the scan ID in the NEXUS_IQ_SCAN_ID environment variable that can be later used by the fecth-sbom action.

GitLab Ultimate customers can use the SBOM file to populate the Dependency List security section, as long as the following conditions are met:

  • the SBOM is in JSON format, and the content conforms to the CycloneDX standard version 1.4, 1.5 or 1.6;
  • the --update-dependency-list flag is present;
  • the SBOM file is set as a pipeline artifact of type: cyclonedx.
Usage
download_sbom:
  needs: [ <evaluate-step-name> ]
  when: always
  stage: <GitLab pipeline stage>
  image: sonatype/gitlab-nexus-iq-pipeline:latest
  script:
    - /sonatype/fetch-sbom [options]

Example

iq_policy_evaluation:
  stage: test
  image: sonatype/gitlab-nexus-iq-pipeline:latest
  script:
    - /sonatype/evaluate -i testapp -r scan-result.json *.jar
  artifacts:
    when: always
    reports:
      dotenv: evaluate.env
    paths:
      - scan-result.json

download_sbom:
  needs: ["iq_policy_evaluation"]
  when: always
  stage: deploy
  image: sonatype/gitlab-nexus-iq-pipeline:latest
  script:
    - /sonatype/fetch-sbom -i testapp -si ${NEXUS_IQ_SCAN_ID} -ss cycloneDx -sv 1.5 -sf json -n sbom.cdx.json -udl
  artifacts:
    reports:
      cyclonedx: sbom.cdx.json
Available Options

-s, --server-url <iq-server-url>

IQ Server URL. If not provided, the NEXUS_IQ_URL environment variable must be set. Required

-a, --authentication <username:password>

IQ Server credentials. If not provided, the NEXUS_IQ_USERNAME and NEXUS_IQ_PASSWORD environment variables must be set. Required

-i, --application-id <iq-application-id>

The ID of the application on the IQ Server. Required

-si, --scan-id <iq-scan-id>

The scan ID of the SBOM file to be downloaded. The NEXUS_IQ_SCAN_ID environment variable case be used as a value. Required

-ss, --sbom-standard <sbom-standard>

SBOM standard. Accepted values: cycloneDx, spdx. Required

-sv, --sbom-version <sbom-version

SBOM version. Accepted values for cycloneDx: 1.2, 1.3, 1.4, 1.5, 1.6; for spdx: 2.3. Required

-sf, --sbom-format <sbom-format>

SBOM format. Accepted values: json, xml. Default: json

-n, --artifact-name <sbom-file-name>

The name of the SBOM file stored as a pipeline artifact. Default: sbom.{sbom-standard}.{sbom-format}

-udl, --update-dependency-list

GitLab Ultimate: Enhance the SBOM with metadata needed for the Dependency List feature. Default: false

Release Notes

Release notes are available here.

Docker Pull Command

docker pull sonatype/gitlab-nexus-iq-pipeline