this isn't used!!!! (see haproxy)
There are two example compose files which should explain how this works.
Docker networking is a little confusing. An ip added to the overlay network outside of docker cannot be accessed from any other node. What this means is that we need to run per node instances of the lb with the same ip.
I left an example for linking lb instances here so that once this shortcoming of the overlay network is fixed, we can have truely redundant load balancers across nodes. with all workers speaking to a single lb.
Right now we just shove a single unlinked lb on each node. They ALL grab the defined ip (which doesn't conflict because that ip is node specific). The worker ips behind each lb ARE accessible across nodes. Even though the workers on each node are communicating with a node specific ip, the loadbalancer is distributing that request to ALL of the workers on all nodes that are running that service.
In production, we will probably run 2 linked loadbalancers on each node. So if something goes wrong with one container on that node, the other will pick up the ip and start handling requests. That is a little silly, but is lightweight and better safe than sorry.
configuration is handled by the following ENV variables.
CONSUL_URL=10.0.0.113:8500 - the ip and port of the consul server (or consul load balancer)
SERVICE=postfix-test - the service is the shortest possible version of the image of the containers that we want to grab ports for and load balance.
ROUTER_ID=MM_POSTFIX_TEST_2 - this should be unique across running instances
KEEPALIVED_STATE=MASTER - state lb tries to pick up on boot. only one server can be master if you want linking to work. for non linked, it does't matter
KEEPALIVED_UNICAST=0 - unicast is the magic behind linking... without it, the instances can't communicate across nodes... so it is pretty magic.
KEEPALIVED_IFACE=eth0 - interface on the continer to add ips to and such.
KEEPALIVED_PRIORITY=150 - should be different on master and slaves, but doesn't really matter for unlinked containers
KEEPALIVED_PASS=32849yrfhv9g3 - doesn't matter all that much in our closed system... but if you are doing multicast (which we can't) this will keep hackers from stealing master status from you.
KEEPALIVED_IP=192.168.254.1 - the ip for keepalived to manage.
KEEPALIVED_PORT=25 - the port on that ip that keepalived will forward to services
KEEPALIVED_NETMASK=255.255.0.0 - the netmask for the above ip.
VROUTER_ID=1 - should be different across lbs, but not across master/slave pairs.