Public | Automated Build

Last pushed: 2 years ago
Short Description
HTTP load balancer integrated with Docker
Full Description

Haystack

Haystack is a HTTP load balancer integrated with
docker. As containers start or stop they
are added or removed from the load balancer.

Quick Start

Haystack uses SRV records
to register docker containers. Haystack connects to Docker using the
DOCKER_HOST and DOCKER_CERT_PATH environment variables registering
any existing containers. Haystack continues to regsister (and
unregister) further containers as they stop and start.

Lets try this out, by starting up Haystack in docker:

If you are using TLS (recommended):

docker run -e DOCKER_HOST=${DOCKER_HOST} \
           -e DOCKER_KEY="$(cat ${DOCKER_CERT_PATH}/key.pem)" \
           -e DOCKER_CERT="$(cat ${DOCKER_CERT_PATH}/cert.pem)" \
           --name=haystack \
           --publish=53:53/udp \
           --publish=80:80 \
           --publish=8080:8080 \
           --publish=22022:22 \
           --detach \
           shortishly/haystack

Otherwise without TLS:

docker run -e DOCKER_HOST=${DOCKER_HOST} \
           --name=haystack \
           --publish=53:53/udp \
           --publish=80:80 \
           --publish=8080:8080 \
           --publish=22022:22 \
           --detach \
           shortishly/haystack

You may need to ensure that your firewall is allowing access to this
port. Consult your local documentation, on Fedora you can check
whether access is enabled via:

sudo firewall-cmd --list-ports

When not using TLS it should output something like:

2375/tcp

Where 2375 is the TCP port used by Docker. You can quickly open access
to your Docker daemon via:

sudo firewall-cmd --add-port=2375/tcp

As an example, lets create a pool of nginx
servers - each one is serving HTTP content from its own directory so
that we can see Haystack load balancing over the containers.

mkdir a b c d

echo a>a/index.html
echo b>b/index.html
echo c>c/index.html
echo d>d/index.html

docker run -v $(pwd)/a:/usr/share/nginx/html:ro -d nginx
docker run -v $(pwd)/b:/usr/share/nginx/html:ro -d nginx
docker run -v $(pwd)/c:/usr/share/nginx/html:ro -d nginx
docker run -v $(pwd)/d:/usr/share/nginx/html:ro -d nginx

Sometimes it is necessary to group a bunch of services - lets create a
web group too:

docker run --detach --name web-001 nginx
docker run --detach --name web-002 nginx

Haystack will automatically load balance services that are part of a
group. A group is identified by its name, followed by a dash and then
a number.

Start a busybox terminal session with
Haystack providing the DNS:

docker run --dns=$(docker inspect --format='{{.NetworkSettings.IPAddress}}' haystack) \
           --tty \
           --interactive \
           --rm busybox \
           /bin/sh

Haystack will have automatically registered the nginx servers that we
created earlier. We can confirm this in our busybox session:

nslookup nginx.services.haystack

The DNS service should respond with an IP address for
nginx.services.haystack. Any docker container that exposes a HTTP
endpoint, will be automatically load balanced by Haystack.

We can test this by making a http request to nginx.services.haystack
and seeing which container responds:

# wget -q -O /dev/stdout http://nginx.services.haystack
c
# wget -q -O /dev/stdout http://nginx.services.haystack
a
# wget -q -O /dev/stdout http://nginx.services.haystack
b
# wget -q -O /dev/stdout http://nginx.services.haystack
c
# wget -q -O /dev/stdout http://nginx.services.haystack
b
# wget -q -O /dev/stdout http://nginx.services.haystack
d

You can stop/kill one or more of the nginx containers above and
Haystack will automatically load balance over the remaining
containers. Haystack will remove nginx.services.haystack from its
DNS when the last nginx container is gone.

We can also load balance over the web group that we created earlier
by using:

wget -q -O /dev/stdout http://web.nginx.services.haystack

You'll notice that your HTTP requests are being handled by a different
set of nginx servers compared to those previously above.

Any HTTP service can be automatically load balanced by Haystack. Lets
try some Apache HTTP containers:

docker run --detached httpd

Back in the busybox terminal:

wget http://httpd.services.haystack

Some services use the alt-HTTP port (8080). Lets try running a
Jenkins service:

docker run --detached jenkins

Back again in the busybox terminal:

wget http://jenkins.services.haystack:8080/

Maintenance

Haystack runs a SSHD for maintenance or debugging the system. To
access Haystack you should add your public key to the authorised keys
that Haystack accepts.

ssh -p 22022 \
    $(docker inspect --format='{{.NetworkSettings.IPAddress}}' haystack)

If you are using docker-machine:

ssh -p 22022 $(docker-machine ip dev)

Where dev is the name of your docker-machine environment that you
are using.

Tracing for Haystack can be enabled via:

haystack:trace(true).

Using false will disable tracing.

Use exit() to exit from the Haystack shell.

Building

Haystack uses erlang.mk. To build run:

make

Running

To run the release:

make run
Docker Pull Command
Owner
shortishly
Source Repository