provartesting/dx-cli

By provartesting

•Updated about 1 month ago

ProvarDx CLI is used to execute provar tests using provardx.

Image

Integration & delivery

Developer tools

1.9K

Overview Tags

provartesting/dx-cli repository overview

⁠Public Docker Image for ProvarDx, hosted on Docker Hub.

⁠Purpose

To provide a pre-configured environment for running Provar tests using ProvarDx with essential software and dependencies.

⁠Included Components

Provar ANT CLI
Java 21
Node.js 18
Salesforce CLI
ProvarDX
Chrome and Firefox browsers

⁠Prerequisites

Provar project on a source control repository.
Presence of .licenses (required) and .smtp (optional) at the project location.
provardx-properties.json file link⁠.
Salesforce org with DevHub enabled.
Recommended: CI job setup.

⁠Usage

Choose Provar Image with a particular tag.
Create a Dockerfile in your project directory.
Define the Dockerfile with base image, configurations, and dependencies.

⁠We have various scenarios available for conducting tests using provardx.

Execute tests on DevHub Org using ProvarDX
Execute tests on the scratch org using provardx
Execute tests on the scratch org using provardx and copy metadata from the DevHub org to the scratch org.

⁠Scenario 1: Execute Tests on Dev Hub Org using ProvarDX

Example Dockerfile

## Provar base tag name
ARG PROVAR_VERSION
FROM provartesting/dx-cli:<PROVAR_VERSION>
ARG browser
ARG PROJECT_PATH
ARG PROJECT_HOME
ARG JWT_KEY
ARG CLIENT_ID
ARG ORG_USER
ENV BROWSER=${browser} \
    PROJECT_PATH=${PROJECT_PATH} \
    PROJECT_HOME=${PROJECT_HOME} \
    RESULT_PATH=${PROJECT_PATH}/DF/ANT/Results \
    JAVA_ARGS=-verbose: class \
    DISPLAY=:99.0 \
    JWT_KEY=${JWT_KEY} \
    JAVA_OPTS="-Djava.awt.headless=true" \
    CLIENT_ID=${CLIENT_ID} \
    ORG_USER=${ORG_USER}
COPY . ${PROJECT_PATH}
COPY License* ${PROVAR_HOME}/.licenses/
COPY smtp1.config ${PROVAR_HOME}/.smtp/
WORKDIR ${PROJECT_PATH}

RUN chmod +x ./entrypoint.sh
ENTRYPOINT ["./entrypoint.sh"]
CMD

Example Entrypoint.sh

#!/bin/sh

cd ${PROJECT_HOME}/provardx
export DISPLAY=:1
Xvfb :1 -screen 0 1024x768x16 &
sfdx provar:validate
sfdx provar:runtests --provarhome=${PROVAR_HOME} -p provardx-properties.json

Example provardx-properties.json: Don’t add “connectionOverride” in provardx-properties.json.

{
	"provarHome": "${PROVAR_HOME}",
	"projectPath": "${PROJECT_PATH}/${PROJECT_HOME}",
	"resultsPath": "${RESULT_PATH}",
	"smtpPath": "",
	"resultsPathDisposition": "Increment",
	"testOutputLevel": "BASIC",
	"pluginOutputlevel": "WARNING",
	"stopOnError": false,
	"lightningMode": true,
	"connectionRefreshType": "Reload",
	"metadata": {
		"metadataLevel": "Reuse",
		"cachePath": "../.provarCaches"
	},
	"environment": {
		"testEnvironment": "",
		"webBrowser": "Chrome",
		"webBrowserConfig": "Full Screen",
		"webBrowserProviderName": "Desktop",
		"webBrowserDeviceName": "Full Screen"
	},
	"testcase": [
		"/test1.testcase"
	]
}

In case we already have our scratch org, to execute tests on it we can use connectionOverride in our json file as shown below.

"connectionOverride": [
        {
            "connection": "<connection_to_override>",
            "username": "<scratch_org>"
        }
    ]

⁠Scenario 2: Consider we want to execute our tests on the scratch org using provardx. We are keeping scratch org as it is.

Example Dockerfile

## Provar base tag name
ARG PROVAR_VERSION
FROM <provardx-image>:<PROVAR_VERSION>
ARG browser
ARG PROJECT_PATH
ARG PROJECT_HOME
ARG JWT_KEY
ARG CLIENT_ID
ARG ORG_USER
ENV BROWSER=${browser} \
    PROJECT_PATH=${PROJECT_PATH} \
    PROJECT_HOME=${PROJECT_HOME} \
    RESULT_PATH=${PROJECT_PATH}/DF/ANT/Results \
    JAVA_ARGS=-verbose: class \
    DISPLAY=:99.0 \
    JWT_KEY=${JWT_KEY} \
    JAVA_OPTS="-Djava.awt.headless=true" \
    CLIENT_ID=${CLIENT_ID} \
    ORG_USER=${ORG_USER}
COPY . ${PROJECT_PATH}
COPY License* ${PROVAR_HOME}/.licenses/
COPY smtp1.config ${PROVAR_HOME}/.smtp/
WORKDIR ${PROJECT_PATH}
RUN apt-get install -y jq dnsutils
RUN chmod +x ./entrypoint.sh
ENTRYPOINT ["./entrypoint.sh"]
CMD

Example entrypoint.sh

#! /bin/bash
#
# Requires jq - see https://stedolan.github.io/jq/
#
# Assumes that the following env vars are set:
#
# CLIENT_ID - the client ID for your OAuth 2 app
# JWT_KEY - path to the private key for creating a JWT
# ORG_USER - authentication username
scratchOrg="dockerScratch"
PROJECT_NAME_ORG="ProvarCI"
# JWT login
sfdx org login jwt --username=${ORG_USER} --jwt-key-file=${JWT_KEY} --client-id=${CLIENT_ID} --set-default-dev-hub
sfdx force:project:create -n ${PROJECT_NAME_ORG}
cd ${PROJECT_NAME_ORG}/config
# Create an org
sf org create scratch -f project-scratch-def.json --edition developer --alias ${scratchOrg} --admin-email <admin-email-id>

# Generate user password
password="$(sfdx force:user:password:generate -u ${scratchOrg} --json | jq -r .result.password)"

# Get accessToken, instanceUrl etc
org="$(sfdx force:org:display -u ${scratchOrg} --json)"

orgId="$(echo ${org} | jq -r .result.id)"
username="$(echo ${org} | jq -r .result. Username)"

accessToken="$(echo ${org} | jq -r .result.accessToken)"
instanceUrl="$(echo ${org} | jq -r .result.instanceUrl)"

# Get my IP address
myip="$(curl -s http://ipinfo.io/ip)"

echo $myip

# Set IP range
opened="$(curl -s ${instanceUrl}/services/Soap/m/39.0/${orgId} \
  -H "Content-Type: text/xml; charset=UTF-8" \
  -H "SOAPAction: updateMetadata" \
  -d '<?xml version="1.0" encoding="UTF-8"?>
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <env:Header>
    <SessionHeader xmlns="http://soap.sforce.com/2006/04/metadata">
      <sessionId>'${accessToken}'</sessionId>
    </SessionHeader>
  </env:Header>
  <env:Body>
    <m:updateMetadata xmlns:m="http://soap.sforce.com/2006/04/metadata" xmlns:sobj="null">
      <m:metadata xsi:type="m:SecuritySettings">
        <m:networkAccess>
          <m:ipRanges>
            <m:start>'${myip}'</m:start>
            <m:end>'${myip}'</m:end>
          </m:ipRanges>
        </m:networkAccess>
      </m:metadata>
    </m:updateMetadata>
  </env:Body>
</env:Envelope>')"

# Move to provardx folder
cd ${PROJECT_PATH}/${PROJECT_HOME}/provardx
# validate the json file
sfdx provar:validated
export DISPLAY=:1
# Initiate Display to execute browser
Xvfb :1 -screen 0 1024x768x16 &
# Runtest using provardx
sfdx provar:runtests --provarhome=${PROVAR_HOME} -p provardx-properties.json
# Delete the scratch org
echo y | sf org delete scratch --target-org ${scratchOrg}

Example provardx-properties.json: Add “connectionOverride” in provardx-properties.json.

{
	"provarHome": "${PROVAR_HOME}",
	"projectPath": "${PROJECT_PATH}/${PROJECT_HOME}",
	"resultsPath": "${RESULT_PATH}",
	"smtpPath": "",
	"resultsPathDisposition": "Increment",
	"testOutputLevel": "BASIC",
	"pluginOutputlevel": "WARNING",
	"stopOnError": false,
	"lightningMode": true,
	"connectionRefreshType": "Reload",
	"metadata": {
		"metadataLevel": "Reuse",
		"cachePath": "../.provarCaches"
	},
	"environment": {
		"testEnvironment": "",
		"webBrowser": "Chrome",
		"webBrowserConfig": "Full Screen",
		"webBrowserProviderName": "Desktop",
		"webBrowserDeviceName": "Full Screen"
	},
	"testCase": [
		"/test1.testcase"
	],
    "connectionOverride": [
        {
            "connection": "<connection_to_override>",
            "username": "<scratch_org>"
        }
    ]

}

⁠Scenario 3: Consider we want to execute our tests on the scratch org using provardx and copy metadata from the original org to the scratch org.

Example Dockerfile: Dockerfile remains same as created in Scenario 1.

Example entrypoint.sh

#!/bin/sh
scratchOrg="dockerScratch"
PROJECT_NAME_ORG="ProvarCI"


# Create a project
sfdx force:project:create -n ${PROJECT_NAME_ORG}
# Login into the org
sfdx org login jwt --username=${ORG_USER} --jwt-key-file=${JWT_KEY} --client-id=${CLIENT_ID} --set-default-dev-hub
cd /home/${PROJECT_NAME_ORG}/config
# Create an org
sf org create scratch -f project-scratch-def.json --edition developer --alias ${scratchOrg} --admin-email <admin-email-id>
# Copying package.xml to sf project
cp /home/package.xml /home/${PROJECT_NAME_ORG}
cd /home/${PROJECT_NAME_ORG}
# Executing package.xml
sfdx force:source:retrieve -x package.xml --targetusername ${ORG_USER}
# Execute deploy
sfdx force:source:push --target-org ${scratchOrg}
# Move to provardx folder
cd ${PROJECT_PATH}/${PROJECT_HOME}/provardx
# validate the json file
sfdx provar:validated
export DISPLAY=:1
# Initiate Display to execute browser
Xvfb :1 -screen 0 1024x768x16 &
# Runtest using provardx
sfdx provar:runtests --provarhome=${PROVAR_HOME} -p provardx-properties.json
# Delete the scratch org
echo y | sf org delete scratch --target-org ${scratchOrg}

Example provardx-properties.json: Add "connectionOverride" in provardx-properties.json.

{
	"provarHome": "${PROVAR_HOME}",
	"projectPath": "${PROJECT_PATH}/${PROJECT_HOME}",
	"resultsPath": "${RESULT_PATH}",
	"smtpPath": "",
	"resultsPathDisposition": "Increment",
	"testOutputLevel": "BASIC",
	"pluginOutputlevel": "WARNING",
	"stopOnError": false,
	"lightningMode": true,
	"connectionRefreshType": "Reload",
	"metadata": {
		"metadataLevel": "Reuse",
		"cachePath": "../.provarCaches"
	},
	"environment": {
		"testEnvironment": "",
		"webBrowser": "Chrome",
		"webBrowserConfig": "Full Screen",
		"webBrowserProviderName": "Desktop",
		"webBrowserDeviceName": "Full Screen"
	},
	"testcase": [
		"/test1.testcase"
	],
    "connectionOverride": [
        {
            "connection": "<connection_to_override>",
            "username": "<scratch_org>"
        }
    ]

}

⁠Explaining Build Arguments in Dockerfile

Build arguments are crucial during the Docker build process, allowing flexibility and customization. They are defined using the ARG instruction in the Dockerfile and can be set during the build process with the --build-arg flag.

⁠Build Arguments:

⁠PROVAR_VERSION:

Description: Determines the version or tag of the Provar base image for the Docker image.
Examples: latest, 2.10.2, 2.11.1, etc.

⁠PROJECT_PATH:

Description: Specifies the path of the project created inside the container where the project is copied.

⁠PROJECT_HOME:

Description: Represents the project home directory containing tests and other folders.

⁠Browser:

Description: Specifies the name of the browser to be used (e.g., Chrome, Chrome_Headless, Firefox).

⁠JWT_KEY:

Description: DevHub server key used for authentication.
Example: server. Key

⁠CLIENT_ID:

Description: Client ID required for authorizing a Dev Hub org.

⁠ORG_USER:

Description: Username of the Dev Hub org.

⁠Configuring CI Pipeline and Build Parameters

⁠Pipeline Configuration Steps:

⁠Step 1: Store Secrets in the Credentials Manager

Store sensitive information like JWT_KEY, CLIENT_ID, and ORG_USER securely in the Credentials Manager.

⁠Step 2: Checkout Code from Source Control Repository

Configure the pipeline to fetch the code from the source control repository.

⁠Step 3: Build Docker Image with Build Arguments

Utilize the Dockerfile to build a Docker image, incorporating build arguments. The parameters can be employed to parameterize the build process. Example build command:

docker build --no-cache -t <your-image-name>:<tag> \
    --build-arg PROVAR_VERSION=$PROVAR_VERSION \
    --build-arg PROJECT_HOME=$PROJECT_HOME \
    --build-arg PROJECT_PATH=$PROJECT_PATH \
    --build-arg browser=$browser \
    --build-arg JWT_KEY=$PROJECT_PATH/server.key \
    --build-arg CLIENT_ID=$CLIENT_ID \
    --build-arg ORG_USER=$ORG_USER .

⁠Step 4: Execute Docker Container for Test Execution

Run the Docker container to initiate test execution, ensuring proper volume mapping to retrieve results. Example command:

docker run --rm -i -v ${WORKSPACE}/Results:${PROJECT_PATH}/${PROJECT_HOME}/Results/ --name provardx <your-image-name>:<tag>

⁠Step 5: Notify Team with Test Report

Implement notification mechanisms such as Slack or email to inform the team about the test results.

⁠Additional Parameters and ChromeOptions Configuration

⁠Additional Parameters

Users can include additional parameters for enhanced control and configuration during test execution:

PROVAR_AUTORETRY_TIMEOUT: Specifies the timeout duration for auto-retry in seconds. Example: PROVAR_AUTORETRY_TIMEOUT=30
PROVAR_AUTORETRY_OVERRIDE: Controls auto-retry behavior. Set to ENABLE_ALL for enabling auto-retry. Example: PROVAR_AUTORETRY_OVERRIDE=ENABLE_ALL

⁠ChromeOptions Configuration

ChromeOptions allow users to configure Chrome browser settings for specific requirements. In this case, it is recommended to add the following ChromeOptions:

"--disable-dev-shm-usage": This option is essential in certain CI system environments or when running inside a Docker instance. It addresses issues where the /dev/shm partition is too small, causing Chrome to fail or crash.

Tag summary

Recent tags

Content type

Image

Digest

sha256:f3806148c…

Size

1.2 GB

Last updated

about 1 month ago

docker pull provartesting/dx-cli