comp1531/basic repository overview

⁠COMP1531 CI infrastructure

The various bits and pieces needed for the CI infrastructure for COMP1531. The submodules are:

• udocker - A patched version of udocker. We use udocker so that CI jobs can at least be somewhat isolated from each other and the class account. It is not hard to break out of a udocker container, so this is not a secure solution. The only reason we need a patched version is the official version always prints an annoying banner on every udocker run which pollutes logs. The patch has been submitted upstream, so hopefully we can use the official version in future.
• udocker-driver - A Gitlab Runner Custom Executor driver for udocker. Currently, this driver assumes the patched udocker version is being used. While it will read the value in image from the .gitlab-ci.yml and try to use it, there are a lot of limitations on the images that can be used. Firstly, the image has to work with udocker. Secondly, the image must contain both git and bash.

⁠Setup

Assuming you're using this to set up a new course, clone this repo into the class account and run:

./setup.sh

Register the runner as normal by first getting the token for a group runner from the gitlab UI and then running the following command, replacing TOKEN with the token and COMPXXXX with the course.

./gitlab-runner register \
  --non-interactive \
  --url "https://gitlab.cse.unsw.edu.au/" \
  --registration-token "TOKEN" \
  --executor "custom" \
  --description "COMPXXXX runner" \
  --access-level="not_protected" \
  --builds-dir="/builds" \
  --cache-dir="/cache" \
  --custom-prepare-exec="$(pwd)/prepare.sh" \
  --custom-run-exec="$(pwd)/run.sh" \
  --custom-cleanup-exec="$(pwd)/cleanup.sh" \
  --output-limit=100

and start it

./gitlab-runner run

That last command should be put in a cron job in case the machine reboots.

⁠Configuration

If the runner is being used for a large course, it may be desirable to increase the number of concurrent jobs the runner can execute from the default of 1. You can do this by setting the concurrent field in ~/.gitlab-runner/config.toml.

⁠Caching

Job caches are stored in /tmp, and udocker images and containers in /var/tmp. This local storage is considerably faster than storing in the class account (via NFS), but it means that for a large course with a lot of concurrent jobs care should be taken not to use too much disk space.