Public Repository

Last pushed: 2 years ago
Short Description
Mesos base image for Joyent Triton
Full Description

Mesos built from https://github.com/joyent/mesos with changes to support Joyent Triton. See all changes and a slide deck explaining them.

This is the base image for the master and slave containers elsewhere.

Architecture

In traditional Mesos deployments, a Mesos slave runs on each physical host, but Triton's approach to containers eliminates the notion of a host and treats the entire data center as a single host. In Triton, a Mesos Slave is used to represent an entire data center, rather than a single host. Multiple data centers can be addressed via multiple slaves to support multi-region deployments.

Each slave represents an entire data center, instead of a single physical host, but the relationship between the slave(s) and the master and schedulers or other Mesos components is unchanged. This has been tested with Marathon, which runs unchanged. We expect that other frameworks on top of Mesos will work unchanged as well.

Overview of changes

Mesos sandbox and Docker volumes

Triton for a variety of reasons has its host volumes as read only and thus, the -v flag linking host volumes has been disabled. In turn we must disable the Mesos-Docker Executor from attempting to add the volume link between the established MESOS_SANDBOX and the docker volumes. We do not however interfere with Mesos establishing a location for the stderr and stdout files (MESOS_SANDBOX) as it is foreseeable that we find a roundabout way of migrating these files into the MESOS_SANDBOX.

Port mapping

Mesos views ports as a consumable resource, however in Triton, where each container gets a unique NIC and IP address, port collisions are impossible and this functionality is no longer warranted. As a result we still track the ports as a resource however whenever a container is created we will not “consume” (remove) the used port from the slaves resources.

Container naming

A minor inconsistency between the Mesos executor and Docker is that the Mesos executor constructs container names by piecing together the executor name with other elements. Unfortunately, the Mesos executor can include capital letters which are not allowed in the names of Docker containers. As a result, illegal names are causing Docker name failures for some requests. The changeset modifies the behavior to coerce container names to fit Docker convention in the executor.

Docker container removal upon destruction

When destroying a Docker container, Mesos would send a docker kill but leave the stopped container in place. Garbage collection issues arose as a result, so we addressed that by removing the container with a docker rm after killing it.

Building

Start with a KVM instance, then login and:

.
apt-get update -q && apt-get install -qy wget && wget -qO- https://get.docker.com/ | sh
git clone git@github.com:joyent/mesos.git && cd mesos
git checkout triton
docker build -t misterbisson/triton-mesos .
Docker Pull Command
Owner
misterbisson

Comments (0)