Puppet Docker image and configuration (manifests) to run a CoreOS
cluster, EPFL-STI style
Docker registry: https://registry.hub.docker.com/u/epflsti/puppet/
- Configure and run the Puppet master: see cluster.foreman
- Check out
docker exec -it puppetmaster.mysubdomain.mydomain.com /bin/bash
git clone https://github.com/epfl-sti/cluster.coreos.puppet.git epflsti_coreos
- Provision a couple of nodes from the Foreman interface; they should auto-integrate into the Puppet cluster.
Build the Puppet agent image with Docker:
docker build -t epflsti/puppet .
Upload the image:
docker push epflsti/puppet
Why Puppet on CoreOS?
At EPFL-STI, we provision bare metal with Foreman. We are currently infatuated with CoreOS, which has its
cloud-config.yaml system, but even they say right there in that page that "[i]t is not intended to be a Chef/Puppet replacement".
In our CoreOS clusters, Puppet + Foreman integration provides us with the following benefits:
- End-to-end automation for node provisioning (i.e. installation and configuration), including IPMI, means you can treat nodes as cattle, not pets
- Assign IP addresses centrally for all use cases (in EPFL-STI clusters we use separate subnets for the following: IPMI for lifecycle management, RFC1918 IPv4 for privileged services, and routable IPv6 for tenants)
- Continuous (re)configuration: add or modify services without reinstalling / rebooting
- Specialized configuration of individual nodes when you really do need it: etcd quorum member, gateway node with the physical Ethernet connection to the outside world...
- Gather key facts (in particular MAC addresses,
dmidecoded serial numbers) into Foreman's centralized database
- Poor man's monitoring: was the node at least alive in the last 30 minutes?
How it works
To run Puppet on CoreOS, we rely on two Docker images:
- one for the Puppetmaster, which is actually bundled with Foreman
- and one for the Puppet agent (see
puppet-agent/Dockerfilein this project).
As is customary for a number of CoreOS services, the Docker container
for the Puppet agent is run as a systemd service, and both the
container and the service are named identically (
In the standard deployment scenario,
cooperate to run the Dockerized Puppet agent a first time after CoreOS
is installed on the node being provisioned, and before it reboots. At
this stage, Puppet is responsible for installing itself into the
on-disk system image so that it operates normally after reboot.
In order for the Puppet code to to distinguish the bootstrap and
steady-state stages, Puppet is passed an environment variable
FACTER_lifecycle_stage=bootstrap, which translates to
$::lifecycle_stage == "bootstrap" in the code. Puppet arranges to
run itself with
FACTER_lifecycle_stage=production after reboot.
This two-stage setup is made necessary by the fact that before
rebooting, the provisioning host needs to transition from the
"building" to "built" states in Foreman; this is so that even if the
BIOS is still configured to boot through PXE, the pxelinux
configuration on Foreman's TFTP server will instruct the provisioned
host to boot from the local disk.
Not all Puppet classes are aware of
$::lifecycle_stage; for those
manifests/init.pp simply excludes them at bootstrap
time. See the comments at the top of the individual
files for more details.