Public | Automated Build

Last pushed: a year ago
Short Description
Containers to support a CI pipeline for puppet modules
Full Description

jenkins_puppetmodule_pipeline

Table of Contents

  1. Overview
  2. Module Description - What the module does and why it is useful
  3. Setup - The basics of getting started with jenkins_puppetmodule_pipeline
  4. Usage - Configuration options and additional functionality
  5. Reference - An under-the-hood peek at what the module is doing and how
  6. Limitations - OS compatibility, etc.
  7. Development - Guide for contributing to the module

Overview

Configures a continuous integration build pipeline in Jenkins for puppet modules.

Module Description

If you are developing puppet modules and want a CI pipeline to automate
building and testing, this module may be of use to you. It uses Jenkins and
publicly available plugins to build a pipeline. There is a puppet type to
declare a pipeline with your module repository. There is also a parameterized
job to build a pipeline by giving any module repository URL.

This module supports Git, Mercurial and Subversion repositories.

Setup

What jenkins_puppetmodule_pipeline affects

  • Jenkins is managed via the rtyler-jenkins module.
  • Docker is installed on the host via the puppetlabs-docker_platform module.

Setup Requirements

  • rbenv and a Ruby version compatible with the modules you will use. rbenv should
    be installed at /opt/rbenv or the RBENV_ROOT system environment variable is
    configured in Jenkins.
  • builder Gem installed.
  • Optionally define GEM_SOURCE as a system environment variable in Jenkins to
    specify where Gems should be found.

Beginning with jenkins_puppetmodule_pipeline

There is an assumption in the documentation that you are familiar with Jenkins
puppet module authoring.

There are Vagrant and Docker containers that you can start with. Checkout the
project and do vagrant up, or pull down the pdouble16/jenkins-puppetmodule-pipeline:jenkins
docker container. Vagrant exposes port 10080, the docker container is using 8080.

Include the jenkins_puppetmodule_pipeline class and go to Jenkins on port
8080 when the puppet run is complete. There is a parameterized job named
AAA-puppetmodule-seed which you can run and enter the repository URL of
your puppet module to create a pipeline. Click on the pipeline folder and then
click Run.

Usage

Pipelines can be specified in puppet with the jenkins_puppetmodule_pipeline::pipeline
defined type.

include jenkins_puppetmodule_pipeline

jenkins_puppetmodule_pipeline::pipeline { 'puppetlabs-stdlib':
    scm_url => 'https://github.com/puppetlabs/puppetlabs-stdlib.git',
}

jenkins_puppetmodule_pipeline::pipeline { 'puppetlabs-concat':
    scm_url => 'https://github.com/puppetlabs/puppetlabs-concat.git',
}

Optional Configuration

The following global environment variables can be set in Jenkins to control the
pipeline:

  • GEM_SOURCE - The Ruby Gems repo, in a form similar to https://rubygems.org/
  • PUPPET_VERSION - The target puppet version, such as 4.4.2

Reference

jenkins_puppetmodule_pipeline

Installs Jenkins, docker, and several plugins needed for the pipeline (see jenkins_puppetmodule_pipeline::plugins)
and a seed project for generating pipelines from a repository URL. If you manage
your own Jenkins server, do not include this class.

jenkins_puppetmodule_pipeline::jenkins

Installs and configures Jenkins. If you already have Jenkins installed by other
means, do not include this class.

jenkins_puppetmodule_pipeline::plugins

Encapsulates the Jenkins plugins necessary for the pipeline. You may include this
class separately if you manage your own Jenkins server, or use it to determine
which plugins to install.

jenkins_puppetmodule_pipeline::docker

Installs and configures docker for beaker acceptance tests. If you already have
docker installed by other means, do not include this class.

jenkins_puppetmodule_pipeline::groovy

Installs and configures groovy that is used for some of the pipeline code, most
notably the deployment jobs.

jenkins_puppetmodule_pipeline::flog

Installs and configures the flog code complexity tool.

jenkins_puppetmodule_pipeline:deploy

Encapsulates the config necessary for the pipeline to deploy modules to test
and production environments. At this time only control repos stored in git
are supported.

jenkins_puppetmodule_pipeline::pipeline, jenkins_puppetmodule_pipeline::pipeline_seed

These classes are the most important.

They produce a Jenkins 'seed' job that creates the pipeline. The jenkins_puppetmodule_pipeline::pipeline
class is the primary use case and takes a parameter for the repository URL for creating the pipeline. Multiple
of these can be specified to ensure the Jenkins server has pipelines for all of
the modules you manage. The jenkins_puppetmodule_pipeline::pipeline_seed creates a
parameterized Jenkins job to create a pipeline with the URL you specify.
The pipeline will be persisted in Jenkins, but won't be managed by puppet.

The pipeline is built of the following jobs:

  • init
  • build
  • sca (Static Code Analysis)
  • acceptance-test jobs for all docker hosts
  • deploy-test
  • deploy-production

init

Checks out the code and configures the rest of the pipeline for building the
module. If there are acceptance tests present using docker as a hypervisor,
a job will be create for each configuration.

build

The following code quality gems are used:

  • ci_reporter_rspec is used to report test results
  • simplecov-rcov is used to report code coverage
  • flog is used to report code complexity

The puppet module does not need to require or otherwise use these gems, the Jenkins
job will add the support as needed.

bundle install --path vendor/bundle
librarian-puppet install --verbose
bundle exec rake --require 'ci/reporter/rake/rspec' build ci:setup:rspec spec

sca

The lint results are reported using the Jenkins warnings plugin.

bundle exec rake check:dot_underscore lint validate

acceptance-test

All host files in spec/acceptance/nodesets using the docker hypervisor will
have jobs created. The execution will look like the following.

export PUPPET_INSTALL_TYPE=agent
export BEAKER_debug=true
export BEAKER_color=no
export BEAKER_set="[operating-system]"
bundle exec rake beaker

In order to test r10k deployment there needs to be a control Git repo. There
is a folder structure in the Jenkins work directory as follows:

  • /var/lib/jenkins/r10k/repos/control/{master,test,production} - the control repo branches
  • /var/lib/jenkins/r10k/repos/modules/ - the root for each module pipeline repo
    that the control repo will reference.

deploy-test

# See pipeline.dsl for code to update metadata.json in the module

# See pipeline.dsl for code to update Puppetfile in the control repo

cd /var/lib/jenkins/r10k/repos/control/test
git pull --rebase origin master
git push origin test

docker exec puppetmaster /opt/rbenv/shims/r10k deploy -v info environment test --puppetfile

deploy-production

cd /var/lib/jenkins/r10k/repos/control/production
git pull --rebase origin test
git push origin production

docker exec puppetmaster /opt/rbenv/shims/r10k deploy -v info environment production --puppetfile

Limitations

  • Requires kernel support for Docker
  • Tested on CentOS 7, Ubuntu 14.04, it'll probably work on much more

Development

Contributions are welcome! The project page is at https://bitbucket.org/double16/jenkins-puppetmodule-pipeline .
Tests are expected where appropriate.

Docker Pull Command
Owner
pdouble16

Comments (0)