HAProxy image that balances between linked containers and, if launched in Docker Cloud or using Docker Compose v2,
reconfigures itself when a linked cluster member redeploys, joins or leaves.
The available version can be found here: https://hub.docker.com/r/dockercloud/haproxy/tags/
latestis built against master branch
stagingis built against staging branch
x.x.xis built against git tags on github
Attention : Please ALWAYS use a specific image tag that works for you. DO NOT use
dockercloud/haproxy:latest in any situation other than testing purpose.
You can use
dockercloud/haproxy in three different ways:
- running in Docker Cloud (Cloud Mode)
- running with Docker legacy links (Legacy Mode)
- running with Docker Compose v2 (Compose Mode, compatible with Docker Swarm)
- running with Docker SwarmMode (Swarm Mode)
Running in Docker Cloud (Cloud Mode)
Launch the service you want to load-balance using Docker Cloud.
Launch the load balancer. To do this, select "Jumpstarts", "Proxies" and select
dockercloud/haproxy. During the "Environment variables" step of the wizard, link to the service created earlier (the name of the link is not important), and add "Full Access" API role (this will allow HAProxy to be updated dynamically by querying Docker Cloud's API).
- If you are using
docker-cloud cli, or
stackfile, please set
- Please DO NOT set
sequential_deployment: trueon this image.
- If you are using
That's it - the haproxy container will start querying Docker Cloud's API for an updated list of containers in the service and reconfigure itself automatically, including:
- start/stop/terminate containers in the linked application services
- start/stop/terminate/scale up/scale down/redeploy the linked application services
- add new links to HAProxy
- remove old links from HAProxy
example of stackfile in Docker Cloud:
web: image: 'dockercloud/hello-world:latest' target_num_containers: 2 lb: image: 'dockercloud/haproxy:latest' links: - web ports: - '80:80' roles: - global
Running with Docker SwarmMode (Swarm Mode)
Docker 1.12 supports SwarmMode natively.
dockercloud/haproxy will auto config itself to load balance all the services running on the same network:
Create a new network using
docker network create -d overlay <name>command
dockercloud/haproxyservice on that network on manager nodes.
Launch your application services that need to be load balanced on the same network.
- You HAVE TO set the environment variable
SERVICE_PORTS=<port1>, <port2>in your application service, which are the ports you would like to expose.
If you mount
/var/run/docker.sock, it can only be run on swarm manager nodes.
If you want the haproxy service to run on worker nodes, you need to setup DOCKER_HOST envvar that points to the manager address.
- You HAVE TO set the environment variable
- If your application services need to access other services(database, for example), you can attach your application services to two different network, one is for database and the other one for the proxy
- This feature is still experimental, please let us know if you find any bugs or have any suggestions.
example of docker swarm mode support
docker network create -d overlay proxy docker service create --name haproxy --network proxy --mount target=/var/run/docker.sock,source=/var/run/docker.sock,type=bind -p 80:80 --constraint "node.role == manager" dockercloud/haproxy docker service create -e SERVICE_PORTS="80" --name app --network proxy --constraint "node.role != manager" dockercloud/hello-world docker service scale app=2 docker service update --env-add VIRTUAL_HOST=web.org app
Running with Docker legacy links (Legacy Mode)
Legacy link refers to the link created before docker 1.10, and the link created in default bridge network in docker 1.10 or after.
example of legacy links using docker cli
docker run -d --name web1 dockercloud/hello-world docker run -d --name web2 dockercloud/hello-world docker run -d -p 80:80 --link web1:web1 --link web2:web2 dockercloud/haproxy
example of docker-compose.yml v1 format:
web1: image: 'dockercloud/hello-world:latest' web2: image: 'dockercloud/hello-world:latest' lb: image: 'dockercloud/haproxy:latest' links: - web1 - web2 ports: - '80:80'
Note: Any link alias sharing the same prefix and followed by "-/_" with an integer is considered to be from the same service. For example:
web-2 belong to service
app_2 are from service
web2 are from different services.
Running with Docker Compose v2 (Compose Mode)
Docker Compose 1.6 supports a new format of the compose file. In the new version(v2), the old link that injects environment variables is deprecated.
Similar to using legacy links, here list some differences that you need to notice:
- This image must be run using Docker Compose, as it relies on the Docker Compose labels for configuration.
- The container needs access to the docker socket, you must mount the correct files and set the related environment to make it work.
- A link is required in order to ensure that dockercloud/haproxy is aware of which service it needs to balance, although links are not needed for service discovery since docker 1.10. Linked aliases are not required.
- DO not overwrite
HOSTNAMEenvironment variable in
- As it is the case on Docker Cloud, auto reconfiguration is supported when the linked services scales or/and the linked container starts/stops.
- The container name is maintained by docker-compose, and used for service discovery as well. Please DO NOT change
container_nameof the linked service in the compose file to a non-standard name. Otherwise, that service will be ignored.
example of docker-compose.yml running on Linux or Docker for Mac (beta):
version: '2' services: web: image: dockercloud/hello-world lb: image: dockercloud/haproxy links: - web volumes: - /var/run/docker.sock:/var/run/docker.sock ports: - 80:80
example of docker-compose.yml running on Mac OS
version: '2' services: web: image: dockercloud/hello-world lb: image: dockercloud/haproxy links: - web environment: - DOCKER_TLS_VERIFY - DOCKER_HOST - DOCKER_CERT_PATH volumes: - $DOCKER_CERT_PATH:$DOCKER_CERT_PATH ports: - 80:80
Once the stack is up, you can scale the web service using
docker-compose scale web=3. dockercloud/haproxy will automatically reload its configuration.
Running with Docker Compose v2 and Swarm (using envvar)
When using links like previous section, the Docker Swarm scheduler can be too restrictive.
Even with overlay network, swarm (As of 1.1.0) will attempt to schedule haproxy on the same node as the linked service due to legacy links behavior.
This can cause unwanted scheduling patterns or errors such as "Unable to find a node fulfilling all dependencies..."
Since Compose V2 allows discovery through the service names, Dockercloud haproxy only needs the links to indentify which service should be load balanced.
A second option is to use the
ADDITIONAL_SERVICES variable for indentification of services.
- Set the
ADDITIONAL_SERVICESenv variable to your linked services.
- You also want to set depends_on to ensure the web service is started before haproxy so that the hostname can be resolved. This controls scheduling order but not location.
- The container still needs access to the docker daemon to get load balanced containers' configs.
- If any trouble with haproxy not updating the config, try running reload.sh or set the
- This image is also compatible with Docker Swarm, and supports the docker native
overlaynetwork across multi-hosts.
example of docker-compose.yml in 'project_dir' directory running in linux:
version: '2' services: web: image: dockercloud/hello-world blog: image: dockercloud/hello-world lb: image: dockercloud/haproxy depends_on: - web - blog environment: - ADDITIONAL_SERVICES=project_dir:web,project_dir:blog volumes: - /var/run/docker.sock:/var/run/docker.sock ports: - 80:80
Global and default settings of HAProxy
Settings in this part is immutable, you have to redeploy HAProxy service to make the changes take effects
|ADDITIONALBACKEND\<NAME\>||add an additional backend with the name set in <NAME>. Possible values include:
|ADDITIONAL_SERVICES||list of additional services to balance (es:
||service]` container labels. This environment variable only works on compose v2, and the referenced services must be on a network resolvable and accessible to this containers.|
|BALANCE||roundrobin||load balancing algorithm to use. Possible values include:
|CA_CERT_FILE||the path of a ca-cert file. This allows you to mount your ca-cert file directly from a volume instead of from envvar. If set,
|CA_CERT||CA cert for haproxy to verify the client. Use the same format as
|CERT_FOLDER||the path of certificates. This allows you to mount your certificate files directly from a volume instead of from envvars. If set,
|DEFAULT_SSL_CERT||Default ssl cert, a pem file content with private key followed by public certificate, '\n'(two chars) as the line separator. should be formatted as one line - see SSL Termination|
|EXTRA_DEFAULT_SETTINGS||comma-separated string of extra settings, and each part will be appended to DEFAULT section in the configuration file. To escape comma, use
|EXTRA_FRONTENDSETTINGS\<PORT\>||comma-separated string of extra settings, and each part will be appended frontend section with the port number specified in the name of the envvar. To escape comma, use
|EXTRA_GLOBAL_SETTINGS||comma-separated string of extra settings, and each part will be appended to GLOBAL section in the configuration file. To escape comma, use
|EXTRA_ROUTE_SETTINGS||a string which is append to the each backend route after the health check, can be over written in the linked services. Possible value: "send-proxy"|
|EXTRA_SSL_CERTS||list of extra certificate names separated by comma, eg.
|FORCE_DEFAULT_BACKEND||True||set the default_service as a default backend. This is useful when you have more than one backend and you don't want your default_service as a default backend|
|HEALTH_CHECK||check||set health check on each backend route, possible value: "check inter 2000 rise 2 fall 3". See:HAProxy:check|
|HTTP_BASIC_AUTH||a comma-separated list of credentials(
|HTTP_BASIC_AUTH_SECURE||a comma-separated list of credentials(
|MAXCONN||4096||sets the maximum per-process number of concurrent connections.|
|MODE||http||mode of load balancing for HAProxy. Possible values include:
|MONITOR_PORT||the port number where monitor_uri should be added to. Use together with
|MONITOR_URI||the exact URI which we want to intercept to return HAProxy's health status instead of forwarding the request.See: http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#4-monitor-uri. Possible value:
|OPTION||redispatch||comma-separated list of HAProxy
|RELOAD_TIMEOUT||0||When haproxy is reconfigured, a new process starts and attaches to the TCP socket for new connections, leaving the old process to handle existing connections. This timeout specifies how long the old process is permitted to continue running before being killed. <br/>
|RSYSLOG_DESTINATION||127.0.0.1||the rsyslog destination to where HAProxy logs are sent|
|SKIP_FORWARDED_PROTO||If set to any value, HAProxy will not add an X-Forwarded- headers. This can be used when combining HAProxy with another load balancer|
|SSL_BIND_CIPHERS||explicitly set which SSL ciphers will be used for the SSL server. This sets the HAProxy
|SSL_BIND_OPTIONS||no-sslv3||explicitly set which SSL bind options will be used for the SSL server. This sets the HAProxy
|STATS_AUTH||stats:stats||username and password required to access the Haproxy stats.|
|STATS_PORT||1936||port for the HAProxy stats section. If this port is published, stats can be accessed at
|TIMEOUT||connect 5000, client 50000, server 50000||comma-separated list of HAProxy
|HAPROXY_USER||haproxy||sets the user of the UNIX sockets to the designated system user name|
|HAPROXY_GROUP||haproxy||sets the group of the UNIX sockets to the designated system group name|
Settings in linked application services
Settings here can overwrite the settings in HAProxy, which are only applied to the linked services. If run in Docker Cloud, when the service redeploys, joins or leaves HAProxy service, HAProxy service will automatically update itself to apply the changes
|APPSESSION||sticky session option, possible value
|BALANCE||load balancing algorithm to use. Possible values include:
|COOKIE||sticky session option. Possible value
|DEFAULT_SSL_CERT||similar to SSL_CERT, but stores the pem file at
|EXCLUDE_PORTS||if set, the application by the application services to the backend routes. You can exclude the ports that you don't want to be routed, like database port|
|EXCLUDE_BASIC_AUTH||if set(any value) and
|EXTRA_ROUTE_SETTINGS||a string which is append to the each backend route after the health check,possible value: "send-proxy"|
|EXTRA_SETTINGS||comma-separated string of extra settings, and each part will be appended to either related backend section or listen session in the configuration file. To escape comma, use
|FAILOVER||if set(any value), it configures this service to be run as HAProxy
|FORCE_SSL||if set(any value) together with ssl termination enabled. HAProxy will redirect HTTP request to HTTPS request.|
|GZIP_COMPRESSION_TYPE||enable gzip compression. The value of this envvar is a list of MIME types that will be compressed. Some possible values:
|HEALTH_CHECK||set health check on each backend route, possible value: "check inter 2000 rise 2 fall 3". See:HAProxy:check|
|HSTS_MAX_AGE||enable HSTS. It is an integer representing the max age of HSTS in seconds, possible value:
|HTTP_CHECK||enable HTTP protocol to check on the servers health, possible value: "OPTIONS * HTTP/1.1\r\nHost:\ www". See:HAProxy:httpchk|
|OPTION||comma-separated list of HAProxy
|SSL_CERT||ssl cert, a pem file with private key followed by public certificate, '\n'(two chars) as the line separator|
|TCP_PORTS||comma separated ports(e.g. 9000, 9001, 2222/ssl). The port listed in
|VIRTUAL_HOST_WEIGHT||an integer of the weight of an virtual host, used together with
|VIRTUAL_HOST||specify virtual host and virtual path. Format:
Swarm Mode only settings:
|SERVICE_PORTS||envvar||comma separated ports(e.g. 80, 8080), which are the ports you would like to expose in your application service. This envvar is swarm mode only, and it is MUST be set in swarm mode|
|`com.docker.dockercloud.haproxy.deactivate=<true||false>`||label||when this label is set to true, haproxy will ignore the service. Could be useful for switching services on blue/green testing|
Check the HAProxy configuration manual for more information on the above.
example of stackfile in Docker Cloud with settings in linked application:
web: image: 'dockercloud/hello-world:latest' target_num_containers: 2 environment: - TCP_PORTS=443 - EXCLUDE_PORTS=22 lb: image: 'dockercloud/haproxy:latest' links: - web ports: - '80:80' roles: - global
Virtual host and virtual path
Both virtual host and virtual path can be specified in environment variable
VIRTUAL_HOST, which is a set of comma separated urls with the format of
|port||80/433||port number of the virtual host. When the scheme is
|/path||virtual path, starts with
examples of matching
|Virtual host||Match||Not match|
|*example.com||www.example.com, example.com, anotherexample.com||www.abc.com|
|*||any website with HTTP|
|https://\*||any website with HTTPS|
|*/path/||example.com/path/, example.org/path/?u=user||example.com/path, example.com/path/abc|
|*/*/path/*||example.com/path/, example.org/abc/path/, example.net/abc/path/123||example.com/path|
|*/path, */path/||example.com/path, example.org/path/|
|example.com:90, https://example.com||example.com:90, https://example.com|
- The sequence of the acl rules generated based on VIRTUAL_HOST are random. In HAProxy, when an acl rule with a wide scope(e.g. *.example.com) is put before a rule with narrow scope(e.g. web.example.com), the narrow rule will never be reached. As a result, if the virtual hosts you set have overlapping scopes, you need to use
VIRTUAL_HOST_WEIGHTto manually set the order of acl rules, namely, giving the narrow virtual host a higher weight than the wide one.
- Every service that has the same VIRTUAL_HOST environment variable setting will be considered and merged into one single service. It may be useful for some testing scenario.
dockercloud/haproxy supports ssl termination on multiple certificates. For each application that you want ssl terminates, simply set
VIRTUAL_HOST. HAProxy, then, reads the certificate from the link environment and sets the ssl termination up.
Attention: there was a bug that if an environment variable value contains "=", which is common in the
SSL_CERT, docker skips that environment variable. As a result, multiple ssl termination only works on docker 1.7.0 or higher, or in Docker Cloud.
SSL termination is enabled when:
- at least one SSL certificate is set, and
VIRTUAL_HOSTis not set, or it is set with "https" as the scheme.
To set SSL certificate, you can either:
DEFAULT_SSL_CERTin the application services linked to HAProxy
The difference between
DEFAULT_SSL_CERT is that, the multiple certificates specified by
SSL_CERT are stored in as cert1.pem, cert2.pem, ..., whereas the one specified by
DEFAULT_SSL_CERT is always stored as cert0.pem. In that case, HAProxy will use cert0.pem as the default certificate when there is no SNI match. However, when multiple
DEFAULT_SSL_CERT is provided, only one of the certificates can be stored as cert0.pem, others are discarded.
The certificate specified in
dockercloud/haproxy or in the linked application services is a pem file, containing a private key followed by a public certificate(private key must be put before the public certificate and any extra Authority certificates, order matters). You can run the following script to generate a self-signed certificate:
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out ca.pem -days 1080 -nodes -subj '/CN=*/O=My Company Name LTD./C=US' cp key.pem cert.pem cat ca.pem >> cert.pem
Once you have the pem file, you can run this command to convert the file correctly to one line:
awk 1 ORS='\\n' cert.pem
Copy the output and set it as the value of
Affinity and session stickiness
There are three method to setup affinity and sticky session:
BALANCE=sourcein your application service. When setting
sourcemethod of balance, HAProxy will hash the client IP address and make sure that the same IP always goes to the same server.
APPSESSION=<value>. use application session t
Is there a way of using external-link dockercloud-haproxy container
Am getting errors in this container
lb-1 | 2016-04-15T07:28:46.499662692Z ERROR:root:can't start new thread
lb-1 | 2016-04-15T07:28:46.499729540Z Traceback (most recent call last):
lb-1 | 2016-04-15T07:28:46.499746820Z File "/usr/bin/dockercloud-haproxy", line 9, in <module>
lb-1 | 2016-04-15T07:28:46.499763940Z load_entry_point('dockercloud-haproxy==1.3', 'console_scripts', 'dockercloud-haproxy')()
lb-1 | 2016-04-15T07:28:46.499779319Z File "/usr/lib/python2.7/site-packages/haproxy/main.py", line 41, in main
lb-1 | 2016-04-15T07:28:46.499794579Z listen_dockercloud_events()
lb-1 | 2016-04-15T07:28:46.499823939Z File "/usr/lib/python2.7/site-packages/haproxy/eventhandler.py", line 60, in listen_dockercloud_events
lb-1 | 2016-04-15T07:28:46.499840468Z events.run_forever()
lb-1 | 2016-04-15T07:28:46.499855468Z File "/usr/lib/python2.7/site-packages/dockercloud/api/events.py", line 42, in run_forever
lb-1 | 2016-04-15T07:28:46.499871128Z ws.run_forever(ping_interval=5, ping_timeout=5, args, *kwargs)
lb-1 | 2016-04-15T07:28:46.499886317Z File "/usr/lib/python2.7/site-packages/websocket/_app.py", line 206, in run_forever
lb-1 | 2016-04-15T07:28:46.499903697Z thread.join()
lb-1 | 2016-04-15T07:28:46.499918657Z File "/usr/lib/python2.7/threading.py", line 929, in join
lb-1 | 2016-04-15T07:28:46.499933987Z raise RuntimeError("cannot join thread before it is started")
lb-1 | 2016-04-15T07:28:46.499949376Z RuntimeError: cannot join thread before it is started
I have found that issue of not reloading automatically if linked containers change is because I had not put:
am using docker-cloud 1.2.1 and it does not automatically configure itself when the linked services changes.
And when I redeploy the container it starts successfully but in between the logs am seeing an warning logs
lb-1 | 2016-03-17T11:18:10.103209709Z INFO:haproxy:===========END===========
lb-1 | 2016-03-17T11:18:51.419971962Z Traceback (most recent call last):
lb-1 | 2016-03-17T11:18:51.420001294Z File "/usr/local/bin/dockercloud-haproxy", line 9, in <module>
lb-1 | 2016-03-17T11:18:51.420010335Z load_entry_point('dockercloud-haproxy==1.2.1', 'console_scripts', 'dockercloud-haproxy')()
lb-1 | 2016-03-17T11:18:51.420019035Z File "/usr/local/lib/python2.7/dist-packages/haproxy/main.py", line 49, in main
lb-1 | 2016-03-17T11:18:51.420025445Z run_haproxy()
lb-1 | 2016-03-17T11:18:51.420031194Z File "/usr/local/lib/python2.7/dist-packages/haproxy/haproxycfg.py", line 25, in run_haproxy
lb-1 | 2016-03-17T11:18:51.420037336Z haproxy.update()
lb-1 | 2016-03-17T11:18:51.420042894Z File "/usr/local/lib/python2.7/dist-packages/haproxy/haproxycfg.py", line 100, in update
lb-1 | 2016-03-17T11:18:51.420048707Z self._update_haproxy(cfg)
lb-1 | 2016-03-17T11:18:51.420054408Z File "/usr/local/lib/python2.7/dist-packages/haproxy/haproxycfg.py", line 120, in _update_haproxy
lb-1 | 2016-03-17T11:18:51.420060125Z UpdateHelper.run_once()
lb-1 | 2016-03-17T11:18:51.420065465Z File "/usr/local/lib/python2.7/dist-packages/haproxy/helper/update_helper.py", line 15, in run_once
lb-1 | 2016-03-17T11:18:51.420071377Z p.wait()
lb-1 | 2016-03-17T11:18:51.420101159Z File "/usr/lib/python2.7/subprocess.py", line 1376, in wait
lb-1 | 2016-03-17T11:18:51.420107979Z pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
lb-1 | 2016-03-17T11:18:51.420113472Z File "/usr/lib/python2.7/subprocess.py", line 476, in _eintr_retry_call
lb-1 | 2016-03-17T11:18:51.420118938Z return func(*args)
lb-1 | 2016-03-17T11:18:51.420124143Z TypeError: exit expected at most 1 arguments, got 2