Reproducible, snapshot-based Debian rootfs builds (especially for Docker).
This is based on lamby's work for reproducible
The name is an attempt at riffing off the photography basis of the word "snapshot". The daguerreotype process was an early method for taking photographs, and this is a method for taking "photographs" of Debian at a given point in time.
The goal is to create an auditable, reproducible process for creating rootfs tarballs (especially for use in Docker) of Debian releases, based on point-in-time snapshots from snapshot.debian.org.
However, as noted below, the only strictly Docker-specific script is
debuerreotype-minimizing-config, which applies many configuration tweaks which are useful for Docker users and may or may not be useful outside of that context.
The usage of the scripts here center around a "rootfs" directory, which is both the working directory for building the target rootfs, and contains the
debuerreotype-epoch file, which records our snapshot.debian.org epoch value (so we can adjust timestamps using it, as it is the basis for our reproducibility).
||create the initial "rootfs", given a suite and a timestamp (in some format
||run a command in the given "rootfs" (using
||apply configuration tweaks to make the rootfs minimal and keep it minimal (especially targeted at Docker images, with comments explicitly describing Docker use cases)|
||remove files such as documentation to create an even smaller rootfs (used for creating
||generate an appropriate
||deterministically create a tar file of the rootfs|
||print out the version of the current
Dockerfile is provided for using these scripts in a simple deterministic environment based on Docker, but given a recent enough version of
debootstrap, they should run fine outside Docker as well (and their deterministic properties have been verified on at least a Gentoo host in addition to the provided Debian-based Docker environment).
Dockerfile also includes comments with hints for bootstrapping the environment on a new architecture (which then presumably doesn't have a
debian Docker base image yet).
Full example: (see
build.sh for this in practice)
$ debuerreotype-init rootfs stretch 2017-01-01T00:00:00Z I: Retrieving InRelease I: Checking Release signature I: Valid Release signature (key id 126C0D24BD8A2942CC7DF8AC7638D0442B90D010) ... I: Checking component main on http://snapshot.debian.org/archive/debian/20170101T000000Z... ... I: Base system installed successfully. $ cat rootfs/debuerreotype-epoch 1483228800 $ debuerreotype-minimizing-config rootfs $ debuerreotype-apt-get rootfs update -qq $ debuerreotype-apt-get rootfs dist-upgrade -yqq $ debuerreotype-apt-get rootfs install -yqq --no-install-recommends inetutils-ping iproute2 debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package libelf1:amd64. (Reading database ... 6299 files and directories currently installed.) Preparing to unpack .../0-libelf1_0.166-2.2_amd64.deb ... Unpacking libelf1:amd64 (0.166-2.2) ... Selecting previously unselected package libmnl0:amd64. Preparing to unpack .../1-libmnl0_1.0.4-2_amd64.deb ... Unpacking libmnl0:amd64 (1.0.4-2) ... Selecting previously unselected package iproute2. Preparing to unpack .../2-iproute2_4.9.0-1_amd64.deb ... Unpacking iproute2 (4.9.0-1) ... Selecting previously unselected package netbase. Preparing to unpack .../3-netbase_5.3_all.deb ... Unpacking netbase (5.3) ... Selecting previously unselected package inetutils-ping. Preparing to unpack .../4-inetutils-ping_2%3a1.9.4-2+b1_amd64.deb ... Unpacking inetutils-ping (2:1.9.4-2+b1) ... Setting up libelf1:amd64 (0.166-2.2) ... Processing triggers for libc-bin (2.24-8) ... Setting up libmnl0:amd64 (1.0.4-2) ... Setting up netbase (5.3) ... Setting up inetutils-ping (2:1.9.4-2+b1) ... Setting up iproute2 (4.9.0-1) ... Processing triggers for libc-bin (2.24-8) ... $ debuerreotype-gen-sources-list rootfs stretch http://deb.debian.org/debian http://security.debian.org $ debuerreotype-tar rootfs - | sha256sum 0542bec04135ed60ed5763f0bcf90381d4e5e33786d57aba5aa4b0fc4e43478a - $ # try it! you should get that same sha256sum value!
How much have you verified this?
Well, I ran the scripts across seven explicit architectures (
s390x) and eight explicit suites (
sid) for a timestamp of
2017-05-16T00:00:00Z (where supported, since
oldstable didn't or no longer currently supports some of those architectures), and there were no modifications to any of the tarballs after several runs across several days.
Additionally, Travis runs with a fixed timestamp value across several suites to verify that their checksums are reproducible, as expected.
From time to time, comments in the files generated by
debuerreotype-minimizing-config might change (for example), which would obviously result in a different checksum, but a simple
diffoscope should be sufficient to verify that the change is benign.