Images are now published to the GitHub Container Registry here; https://github.com/orgs/bcgov/packages/container/package/certbot
Automatically update TLS Certificates on OpenShift Routes
To learn more about the Common Services available visit the Common Services Showcase page.
CronJob which will run on a regular schedule for renewing TLS certificates
CronJob will manage all Route objects annotated with the label certbot-managed=trueThe Certbot container image supports an array of environment variables to configure how it will behave. Certbot behavior can be modified by modifying which variables are defined. The following variables change the way how the internal Certbot application will behave.
| Environment Variable | Default Value | Notes |
|---|---|---|
CERTBOT_CONFIG_DIR | /etc/letsencrypt | Certbot Config Directory (Should be backed by a persistent volume) |
CERTBOT_DEPLOY_DIR | /etc/letsencrypt/renewal-hooks/deploy | Certbot Deploy Directory |
CERTBOT_LOGS_DIR | /var/log/letsencrypt | Certbot Log Directory |
CERTBOT_WORK_DIR | /var/lib/letsencrypt | Certbot Working Directory |
CERTBOT_DEBUG | false | Enable Certbot debug logging |
CERTBOT_DELETE_ACME_ROUTES | true | Automatically clean up temporary ACME challenge routes on completion |
CERTBOT_DRY_RUN | false | Performs a mock Certbot execution for CSR signability |
CERTBOT_EMAIL | Correspondence email to register with Certificate Authority | |
CERTBOT_RSA_KEY_SIZE | 2048 | Key length for RSA keypair generation |
CERTBOT_STAGING | false | Use self-signed cert renewals. Must be false if using Entrust) |
CERTBOT_SUBSET | true | Allow Certbot to pass ACME challenge if at least one domain succeeds |
The following provides you a quick way to get Certbot set up and running as an OpenShift cronjob.
Point to the appropriate project/namespace on OpenShift
export NAMESPACE=<YOURNAMESPACE>
oc project $NAMESPACE
Ensure that the Routes you want Certbot to manage have been annotated with the label certbot-managed=true. You can list routes that meet this criteria with the following:
oc get route -n $NAMESPACE -l certbot-managed=true -o=jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}'
Install Certbot to your project/namespace by processing certbot.dc.yaml to create the CronJob and supporting objects (ServiceAccount, RoleBinding, PVC, etc).
This template accepts the following parameters which loosely corresponds with the Environment Variables mentioned above (add with -p to the oc process command):
| Parameter | Default Value | Description |
|---|---|---|
CERTBOT_DEBUG | false | Run Certbot in debug mode |
CERTBOT_DELETE_ACME_ROUTES | true | Self cleanup temporary ACME routes when done |
CERTBOT_DRYRUN | false | Run without executing |
CERTBOT_EMAIL | Email where CSR requests are sent to. For Entrust, Product Owner's *@gov.bc.ca is suggested | |
CERTBOT_SERVER | https://acme-v02.api.letsencrypt.org/directory | ACME Certbot endpoint. For BC Gov SSL, see Entrust. |
CERTBOT_STAGING | false | Use self-signed cert renewals. Must be false if using Entrust) |
CERTBOT_SUBSET | true | Allow domain validation to pass if a subset of them are valid |
CRON_SCHEDULE | 0 0 * * 1,4 | Cronjob Schedule |
CRON_SUSPEND | false | Suspend cronjob |
IMAGE_REGISTRY | docker.io | Image Registry |
IMAGE_NAMESPACE | bcgovimages | Image Namespace |
IMAGE_NAME | certbot | Image Name |
IMAGE_TAG | latest | Image Tag. We recommend pinning this to a specific release veresion for stability |
CERTBOT_STAGING=true, so you don't hit any service limits with LetsEncrypt.https://acme-staging-v02.api.letsencrypt.org/directory to avoid being rate limited.CERTBOT_SUBSET option is set to true. Otherwise certificate renewals will always fail because their extra domain will never be managed on our end and choke. If you require stringent domain validation, set CERTBOT_SUBSET to false explicitly.export CERTBOT_EMAIL=<[email protected]>
export CERTBOT_SERVER=<YOURCERTBOTSERVER>
oc process -n $NAMESPACE -f "https://raw.githubusercontent.com/BCDevOps/certbot/master/openshift/certbot.dc.yaml" -p CERTBOT_EMAIL=$CERTBOT_EMAIL -p CERTBOT_SERVER=$CERTBOT_SERVER | oc apply -n $NAMESPACE -f -
PS: You MUST supply a valid email address!
If you need to run the CronJob manually, you can do that by running:
# Create a Job
oc create job -n $NAMESPACE "certbot-manual-$(date +%s)" --from=cronjob/certbot
# Delete any previous manual Jobs created
# Note: When there are no jobs to delete, you will get an error for oc delete.
oc get job -n $NAMESPACE -o name | grep -F -e '-manual-' | xargs oc delete -n $NAMESPACE
To remove certbot from your namespace, run the following commands. All build related manifests will have a build=certbot label, and all cronjob application related manifests will have an app=certbot label.
export NAMESPACE=<YOURNAMESPACE>
# Delete all manifests generated by certbot.bc.yaml
oc delete all -n $NAMESPACE -l build=certbot
# Delete all manifests generated by certbot.dc.yaml
oc delete cronjob,pvc,rolebinding,sa -n $NAMESPACE -l app=certbot
Entrust is the only approved certificate provider for BC Gov production environments currently. There are a few extra steps required to request certificates from Entrust instead of LetsEncrypt.
Start by creating the deployment config found in the Quick Start section
Modify the CERTBOT_SERVER parameter in the deployment config to use Entrust
| Parameter | Default Value | Description |
|---|---|---|
CERTBOT_SERVER | https://www.entrust.net/acme/api/v1/directory/xx-xxxx-xxxx | Where xx-xxxx-xxxx is the directory ID. This value may vary between different ministry organizations. Please contact your organization to determine this value. |
Make sure CERTBOT_STAGING is set to false. The Entrust server does not have a staging mode
If a Certbot job has previously on the same route using LetsEncrypt server, then you will need to delete the existing PVC. This will remove old Let's Encrypt files and a new PVC will be created on the next step.
Apply the deployment config and run the job manually or by cron trigger. The job logs will display it has failed to obtain the certificates and the route will remain unmodified. This is normal because the certificate request still needs to be approved by your ministry first.
In the CERTBOT_EMAIL inbox you should receive an email from [email protected] containing a Tracking ID.
At this point you need to contact the appropriate group in your ministry to create an iStore order to approve the certificate request. For NRM teams, please have your product owner create a Service Desk request containing the following information.
Once the iStore order has been created and approved, you should receive another email from [email protected] letting you know the request has been approved.
Re-run the job and Certbot should obtain the certificates and install them automatically.
If you are going to setup automatic cert renewals for the first time, backup "Certficate", "Private Key" and "CA Certificate" contents from your route.
List your cron jobs
export NAMESPACE=<YOURNAMESPACE>
oc get -n $NAMESPACE cronjob
To describe your cron job
export NAMESPACE=<YOURNAMESPACE>
oc describe -n $NAMESPACE cronjob/certbot
To see your cron jobs in Openshift Console: Administrator View -> Workloads > Jobs
To access the logs for cron jobs in Openshift Console, check for the latest completed/failed cronjob pods in you pod list (Administrator View -> Workloads > Pods).
If you are seeing errors in the logs and need to troubleshoot, you may use optional parameters CERTBOT_DEBUG and CERTBOT_DELETE_ACME_ROUTES.
export NAMESPACE=<YOURNAMESPACE>
export CERTBOT_SERVER=<YOURCERTBOTSERVER>
export CERTBOT_EMAIL=<[email protected]>
oc process -n $NAMESPACE -f "https://raw.githubusercontent.com/BCDevOps/certbot/master/openshift/certbot.dc.yaml" -p CERTBOT_EMAIL=$EMAIL -p CERTBOT_SERVER=$CERTBOT_SERVER -p CERTBOT_STAGING=false -p CERTBOT_DEBUG=true -p CERTBOT_DELETE_ACME_ROUTES=false | oc apply -n $NAMESPACE -f -
PS: Ensure that you manually delete the ACME Route and Service after you are done troubleshooting and redeploy without the DEBUG and DELETE_ACME_ROUTES options!
If you end up running the setup process multiple times, ensure that you have deleted all the duplicate copies of those cron jobs and only keep the latest one. Or to delete all the certbot jobs and start fresh you can use the below.
export NAMESPACE=<YOURNAMESPACE>
oc get job -n $NAMESPACE -o name | grep -F -e 'certbot' | xargs oc delete
oc get cronjob -n $NAMESPACE -o name | grep -F -e 'certbot' | xargs oc delete
To suspend a cronjob in your namespace, you can use the below patch command.
export NAMESPACE=<YOURNAMESPACE>
oc patch cronjob -n $NAMESPACE certbot -p '{"spec" : {"suspend" : false }}'
To resume a cronjob in your namespace, you can use the below patch command.
export NAMESPACE=<YOURNAMESPACE>
oc patch cronjob -n $NAMESPACE certbot -p '{"spec" : {"suspend" : true }}'
If you need to build Certbot directly on the cluster, you can do so by processing certbot.bc.yaml to create the required build objects. For most situations, this is no longer needed.
This template accepts the following parameters (add with -p to the oc process command):
| Parameter | Default Value | Description |
|---|---|---|
GIT_REF | master | Git Pull Request or Branch Reference (i.e. 'pull/CHANGE_ID/head') |
GIT_URL | https://github.com/BCDevOps/certbot.git | Git Repository URL |
oc process -n $NAMESPACE -f "https://raw.githubusercontent.com/BCDevOps/certbot/master/openshift/certbot.bc.yaml" | oc apply -n $NAMESPACE -f -
Copyright 2018 Province of British Columbia
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Content type
Image
Digest
sha256:dff75a6c8…
Size
94.4 MB
Last updated
almost 4 years ago
docker pull bcgovimages/certbot