augusthome/nodescanner repository overview

⁠august-node-scan

This creates a Docker image which August can use to scan node software for known security risks and unwanted secrets.

This builds on three tools:

• https://docs.npmjs.com/cli/audit⁠
• https://github.com/awslabs/git-secrets⁠
• ./entropyscanner.py

⁠Building and Deploying the Image

Assuming:

1. You're running Docker already
2. You have an account on https://hub.docker.com⁠ with access to the augusthome team
3. You've logged in to that account via docker login.

docker build -t augusthome/nodescanner .
docker push augusthome/nodescanner

Once you push a new image, any Bitbucket pipeline builds which refer to it will use it. HOWEVER it will not cause the Bitbucket pipelines to rebuild just because of the new Docker image.

⁠Usage

To run these node security tests, see the following example, and apply the first step to the bitbucket-pipelines.yml at the root of your project.

The clone: depth: 1 can also help speed up your pulls. Bitbucket defaults to a depth of 50.

image: node:8
# Restrict the depth so that the securty scan doesn't waste time looking at old commits
clone:
    depth: 1

pipelines:
  default:
    - parallel:
      - step:
          name: Node Security Scanner
          # Override image to get one with the scanning software
          image:
            name: augusthome/nodescanner
            # Make sure this matches the user "scanner" with `id -u scanner`
            run-as-user: 1001
          script:
            # Runs in /opt/atlassian/pipelines/agent/build
            - ~/scan_node.sh

      - step:
          name: Tests
          script:
            - echo $APICREDS_JSON | base64 --decode > ~/apicreds.json
            - export APIKEYFILE=~/apicreds.json
            - ./scripts/cibuild.sh

⁠Configuration

You can run this with the default configuration, or you can set these environment variables in your project settings:

Configuration variables (value in parens is default):
FAIL_ON_NPM_AUDIT_SCAN       (1)  : 1
FAIL_ON_AWS_SECRETS          (1)  : 1
FAIL_ON_HIGH_ENTROPY         (0)  : 0
NPM_AUDIT_SKIP_SEVERITIES    (moderate,low)  : moderate,low
ENTROPY_MIN (0-1, <0=>skip)  (0.6)  : 0.6
ENTROPY_BLOCK (-1=>whole line) (-1) : -1
ENTROPY_IGNORE (filenames)   (npm-shrinkwrap.json package-lock.json package.json)  : npm-shrinkwrap.json package-lock.json package.json
You can set these as environment variables in the repo settings at
https://bitbucket.org/august_team/august-rest-api/admin/addon/admin/pipelines/repository-variables).

That last line, with the link, will auto-adjust to fit your bitbucket project and point to the correct repository-variables.

⁠FAIL_ON_*

These control if the build fails because a security scan step fails. Set to 0 to ignore failure at a step and 1 to fail the build if the step fails. For example, the FAIL_ON_HIGH_ENTROPY value is 0 by default because this gives very noisy results (a lot of lines of code look like high entropy secrets).

⁠NPM_AUDIT_*

These are parameters to the npm audit security scanner in this repository, ciaudit.js.

• NPM_AUDIT_SKIP_SEVERITIES is a coma-separated list of severity to ignore in the scan. Be careful using this feature. Keep a careful eye on your exceptions.

⁠ENTROPY_*

These are parameters to the home-brewed entropy scanner in this repository, entropyscanner.py.

• ENTROPY_MIN is a value between 0 and 1, where a higher number means more entropy found within the BLOCK. Setting this to 0 will cause the entropy scan to be skipped. Below about 0.5 you see a lot of code. Even at 0.7 there is still a lot of code, but secrets are easier to see, though at this value, some secrets will be missed.
• ENTROPY_BLOCK is the number of bytes (well, kind of assuming ASCII characters here) over which to calculate entropy. How long does a string need to be to consider it a secret? By default (-1) we look at the whole code line, but 20 is a reasonable limit too.
• ENTROPY_IGNORE is a space-separated list of files to ignore in the scan. It is not smart enough to parse wildcards, but it will ignore these files no matter which directory they are in.

⁠git-secrets

This script does not enable configuring git-secrets via environment variables. If you need to add configuration please see https://github.com/awslabs/git-secrets⁠ .