iute is an all-in-one distributed docker registry with next to no coding involved.
iute comes bundled with:
- etcd, a distributed key-value store for locking and discovery
- syncthing, open-source bittorrent file syncing software
- nginx, a reverse-proxy with auth capabilities
- docker distribution, the "docker registry" in registry:2
At ParseHub we wanted a way to reduce our
docker push to
docker pull round trip time, which was bottlenecked on a single build server which pushed to a single registry which was pulled from O(n) servers at once.
Instead, we came up with a system where a cluster of iute registries were running throughout our production and staging machines.
This uses the powerful BitTorrent file distribution system to make the
docker push speed equal to pushing locally,
docker pull speed equivalent to pulling locally.
We use etcd to implement a distributed Readers-writer lock to ensure that no registries are being pulled from at the same time that a registry is being pushed to.
This is a reasonable compromise in our use case, which is
- Clone this repository with
git clone https://github.com/parsehub/iute.git
- Build the docker image with
docker build -t 'iute:latest' .
- Distribute iute to the first host, then
docker run --rm -i iute:latest --create localhost:10000-10006 > bundle.ut,
load the bundle with
docker run --name iute -p 10001-10006:1-6,8000:8000 -i iute:latest:10001-10006 --registry-port 8000 localhost < bundle.ut, and
start iute with
docker start iute
- Copy the bundle over to all servers which are to be part of the cluster,
then repeat the former two commands (with potentially modified ports) to load the bundle and start iute on all of
Local usage example (two registries on same host)
docker rm -f iute docker build -t iute:1.0 . && \ docker run --rm -i iute:1.0 --create localhost:10000-10005 > bundle.ut && \ docker run -p 10000-10005:10000-10005 -p 8000:8000 --network host --name iute -i iute:1.0 localhost:10000-10005 -r 8000 < bundle.ut && \ docker start iute docker rm -f iute2 docker run -p 10006-10011:10006-10011 -p 8000:8000 --network host --name iute2 -i iute:1.0 localhost:10006-10011 < bundle.ut && \ docker start iute2