headmelted/cobbler repository overview

⁠Cobbler

⁠Simpler cross-compilation and testing for busy developers

WARNING: THIS PROJECT IS STILL VERY MUCH A WORK-IN-PROGRESS AND SHOULD NOT BE USED FOR YOUR OWN PROJECTS YET. THERE'S A LOT OF PARTS MOVING AROUND. YE'VE BEEN WARNED!!

⁠What is Cobbler?

Cobbler is a series of pre-configured Debian Stretch docker containers that are collectively intended to make porting, compiling and testing code written for GNU compilers on multiple architectures as simple as possible.

By using some sensible defaults, and several commonly used and well-supported tools, Cobbler's goal is to make compiling platform-agnostic code in several languages to architectures such as ARM, PowerPC and SPARC as simple as building for Intel.

⁠Who should use Cobbler?

Certain projects are a better fit than others for the assumptions Cobbler makes. Specifically, any code that relies heavily on architecture-specific calls is will see limited benefit from using Cobbler, whereas platform-agnostic code (or code with platform-agnostic native dependencies) is likely to have much better results.

It's expected that Cobbler should be able to compile fully platform-agnostic code to any target it supports without any changes, but any architecture-specific calls in your code will need to be patched to support your target. In these cases, Cobbler can still help with migrating your project to support multiple architectures, but it isn't a silver bullet for deeply architecture-dependent code.

⁠How to get started

Typically, the easiest way to get started with Cobbler is to first try to migrate your existing build script to building inside Cobbler with the amd64 target (i.e. an amd64 to amd64 build). Once your scripts are correctly configured for Cobbler, and your project is compiling successfully for amd64, targeting other architectures should be as simple as adding a different target architecture from the Supported Architectures table to your configuration.

⁠Strategies

Cobbler supports three different strategies for working with a target architecture within a session, as explained below. Each of these strategies are agnostic as to the structure of your code, such that you should be able to change the setting as required by your project without making changes to your own code.

⁠cross

The session is executed using amd64 GNU cross-compilers for the target architecture. When using this strategy, the dependency packages for the target architecture are installed, and linked explicitly during compilation.

This is the fastest strategy for compilation (broadly equivalent in performance to compiling directly to amd64 in the simplest cases), but may not play well with all dependencies of your project. During compilation, the $CC and $CXX variables commonly used to alias GCC are modified to explicitly include linking of the supporting dependencies for the target architecture. While this is often all you'll need for cross-compilation, you may experience execution format errors that are likely a sign of native code (specific to the target architecture) trying to run during installation of a dependency. If this happens you'll need to choose another strategy.

⁠hybrid

The session is executed similarly to the cross strategy, but depedencies are installed within an emulated environment for the target architecture such that any native code called during installation of the dependencies will execute as expected. This is typically less performant than the cross method, but increases compatibility by emulating every call not made by the compiler.

This is likely to be suitable for most cross-compilation scenarios, and would be the first thing to try if you encounter exec format errors with the cross strategy. Compilation performance is mostly preserved as amd64-native compilers are still used, and emulation will only occur for dependency installation processes.

⁠emulate

The session is executed inside an emulated environment for the target architecture. This is less performant than using the hybrid strategy, but increases compatibility further by transparently emulating every call.

It may be necessary to switch from hybrid to emulate in certain cases where hard-coded logic inside your code or dependencies requires the target architecture specifically.

⁠virtualize

The session is executed in a virtualized QEMU system that is running an image compiled explicitly for the target architecture. This strategy has the highest compatibility, as all logic will be performed within the target architecture with effectively no trace of native host components - but at a significant performance cost due to the emulation overhead.

This strategy is most useful in scenarios where you need to be able to confirm that your code can be compiled by downstream users with specific devices (e.g. specific niche processors, lower memory). An example of this is targeting the Raspberry Pi line of single-board computers, which can be completely virtualised within a session.

⁠Support matrix

⁠Variables

Cobbler sets a series of global variables inside each session that can be used to help with your builds. The table below explains these variables, and gives some context as to what each one means. Note that Cobbler globals are always prefixed with COBBLER_ so as to prevent conflicts with your own variables except in limited instances where Cobbler intentionally overwrites global variables to make cross-compilation easier, these variables are in bold.

⁠Architecture

There are effectively two lists of supported architectures for Cobbler. Compiling and testing of programs without dependendent packages (i.e. programs for which no dependencies need to be pulled from Debian repositories) is supported for the intersection of architectures of GCC and QEMU.

As Cobbler is built on Debian Stretch, the architectures supported for those programs which have dependencies on standard packages within the repository mirror those supported by the operating system. To clarify, the table below shows the state of support for different architectures within Cobbler, and the level of support that's available in terms of APT packages for that architecture.

** The amd64 target does not actually involve cross-compilation, and simply maps directly to the x86_64 GCC compilers. The target is included so that the the developer's build process can treat amd64 as agnostically as other architectures in Cobbler.