Sign inSign up

maauso/docker-consul-template

By maauso

Updated over 9 years ago

Consul-template 0.16.0 using in maauso/haproxy

Image
1

2.3K

maauso/docker-consul-template repository overview

Consul Template

This project provides a convenient way to populate values from [Consul][] into the file system using the consul-template daemon.

The daemon consul-template queries a [Consul][] instance and updates any number of specified templates on the file system. As an added bonus, consul-template can optionally run arbitrary commands when the update process completes. See the Examples folder for some scenarios where this functionality might prove useful.

The documentation in this README corresponds to the master branch of Consul Template. It may contain unreleased features or different APIs than the most recently released version. Please see the Git tag that corresponds to your version of Consul Template for the proper documentation.

Installation

You can download a released consul-template artifact from the Consul Template release page. If you wish to compile from source, please see the instructions in the Contributing section.

Usage

Options

For the full list of options that correspond with your release, run:

consul-template -h
Command Line

The CLI interface supports all of the options detailed above.

Query the demo.consul.io Consul instance, rendering the template on disk at /tmp/template.ctmpl to /tmp/result, running Consul Template as a service until stopped:

$ consul-template \
  -consul demo.consul.io \
  -template "/tmp/template.ctmpl:/tmp/result"

Query a local Consul instance, rendering the template and restarting Nginx if the template has changed, once, polling 30s if Consul is unavailable:

$ consul-template \
  -consul 127.0.0.1:8500 \
  -template "/tmp/template.ctmpl:/var/www/nginx.conf:service nginx restart" \
  -retry 30s \
  -once

Query a Consul instance, rendering multiple templates and commands as a service until stopped:

$ consul-template \
  -consul my.consul.internal:6124 \
  -template "/tmp/nginx.ctmpl:/var/nginx/nginx.conf:service nginx restart" \
  -template "/tmp/redis.ctmpl:/var/redis/redis.conf:service redis restart" \
  -template "/tmp/haproxy.ctmpl:/var/haproxy/haproxy.conf"

Query a Consul instance that requires authentication, dumping the templates to stdout instead of writing to disk. In this example, the second and third parameters to the -template option are required but ignored. The file will not be written to disk and the optional command will not be executed:

$ consul-template \
  -consul my.consul.internal:6124 \
  -template "/tmp/template.ctmpl:/tmp/result:service nginx restart"
  -dry

Query a Consul that uses custom SSL certificates:

$ consul-template \
  -consul 127.0.0.1:8543 \
  -ssl \
  -ssl-cert /path/to/client/cert.pem \
  -ssl-ca-cert /path/to/ca/cert.pem \
  -template "/tmp/template.ctmpl:/tmp/result" \
  -dry \
  -once

Query Consul and act as a supervisor for the given child process:

$ consul-template \
  -template "/tmp/in.ctmpl:/tmp/result" \
  -exec "/sbin/my-server"

For more information, please see the Exec Mode documentation.

Configuration File(s)

The Consul Template configuration files are written in [HashiCorp Configuration Language (HCL)][HCL]. By proxy, this means the Consul Template configuration file is JSON-compatible. For more information, please see the [HCL specification][HCL].

The Configuration file syntax interface supports all of the options detailed above, unless otherwise noted in the table.

// This is the address of the Consul agent. By default, this is 127.0.0.1:8500,
// which is the default bind and port for a local Consul agent. It is not
// recommended that you communicate directly with a Consul server, and instead
// communicate with the local Consul agent. There are many reasons for this,
// most importantly the Consul agent is able to multiplex connections to the
// Consul server and reduce the number of open HTTP connections. Additionally,
// it provides a "well-known" IP address for which clients can connect.
consul = "127.0.0.1:8500"

// This is the ACL token to use when connecting to Consul. If you did not
// enable ACLs on your Consul cluster, you do not need to set this option.
//
// This option is also available via the environment variable CONSUL_TOKEN.
token = "abcd1234"

// This is the signal to listen for to trigger a reload event. The default
// value is shown below. Setting this value to the empty string will cause CT
// to not listen for any reload signals.
reload_signal = "SIGHUP"

// This is the signal to listen for to trigger a core dump event. The default
// value is shown below. Setting this value to the empty string will cause CT
// to not listen for any core dump signals.
dump_signal = "SIGQUIT"

// This is the signal to listen for to trigger a graceful stop. The default
// value is shown below. Setting this value to the empty string will cause CT
// to not listen for any graceful stop signals.
kill_signal = "SIGINT"

// This is customization around the environment in which template commands are
// executed. See the "exec" block for more information on the specific
// configuration options.
env {
  // ...
}

// This is the amount of time to wait before retrying a connection to Consul.
// Consul Template is highly fault tolerant, meaning it does not exit in the
// face of failure. Instead, it uses exponential back-off and retry functions to
// wait for the cluster to become available, as is customary in distributed
// systems.
retry = "10s"

// This is the maximum interval to allow "stale" data. By default, only the
// Consul leader will respond to queries; any requests to a follower will
// forward to the leader. In large clusters with many requests, this is not as
// scalable, so this option allows any follower to respond to a query, so long
// as the last-replicated data is within these bounds. Higher values result in
// less cluster load, but are more likely to have outdated data.
max_stale = "10m"

// This is the log level. If you find a bug in Consul Template, please enable
// debug logs so we can help identify the issue. This is also available as a
// command line flag.
log_level = "warn"

// This is the path to store a PID file which will contain the process ID of the
// Consul Template process. This is useful if you plan to send custom signals
// to the process.
pid_file = "/path/to/pid"

// This is the quiescence timers; it defines the minimum and maximum amount of
// time to wait for the cluster to reach a consistent state before rendering a
// template. This is useful to enable in systems that have a lot of flapping,
// because it will reduce the the number of times a template is rendered.
wait {
  min = "5s"
  max = "10s"
}

// This denotes the start of the configuration section for Vault. All values
// contained in this section pertain to Vault.
vault {
  // This is the address of the Vault leader. The protocol (http(s)) portion
  // of the address is required.
  address = "https://vault.service.consul:8200"

  // This is the token to use when communicating with the Vault server.
  // Like other tools that integrate with Vault, Consul Template makes the
  // assumption that you provide it with a Vault token; it does not have the
  // incorporated logic to generate tokens via Vault's auth methods.
  //
  // This value can also be specified via the environment variable VAULT_TOKEN.
  token = "abcd1234"

  // This tells Consul Template that the provided token is actually a wrapped
  // token that should be unwrapped using Vault's cubbyhole response wrapping
  // before being used. Please see Vault's cubbyhole response wrapping
  // documentation for more information.
  unwrap_token = true

  // This option tells Consul Template to automatically renew the Vault token
  // given. If you are unfamiliar with Vault's architecture, Vault requires
  // tokens be renewed at some regular interval or they will be revoked. Consul
  // Template will automatically renew the token at half the lease duration of
  // the token. The default value is true, but this option can be disabled if
  // you want to renew the Vault token using an out-of-band process.
  //
  // Note that secrets specified in a template (using {{secret}} for example)
  // are always renewed, even if this option is set to false. This option only
  // applies to the top-level Vault token itself.
  renew_token = true

  // This section details the SSL options for connecting to the Vault server.
  // Please see the SSL options below for more information (they are the same).
  ssl {
    // ...
  }
}

// This block specifies the basic authentication information to pass with the
// request. For more information on authentication, please see the Consul
// documentation.
auth {
  enabled  = true
  username = "test"
  password = "test"
}

// This block configures the SSL options for connecting to the Consul server.
ssl {
  // This enables SSL. Specifying any option for SSL will also enable it.
  enabled = true

  // This enables SSL peer verification. The default value is "true", which
  // will check the global CA chain to make sure the given certificates are
  // valid. If you are using a self-signed certificate that you have not added
  // to the CA chain, you may want to disable SSL verification. However, please
  // understand this is a potential security vulnerability.
  verify = false

  // This is the path to the certificate to use to authenticate. If just a
  // certificate is provided, it is assumed to contain both the certificate and
  // the key to convert to an X509 certificate. If both the certificate and
  // key are specified, Consul Template will automatically combine them into an
  // X509 certificate for you.
  cert = "/path/to/client/cert"
  key = "/path/to/client/key"

  // This is the path to the certificate authority to use as a CA. This is
  // useful for self-signed certificates or for organizations using their own
  // internal certificate authority.
  ca_cert = "/path/to/ca"

  // This is the path to a directory of PEM-encoded CA cert files. If both
  // `ca_cert` and `ca_path` is specified, `ca_cert` is preferred.
  ca_path = "path/to/certs/"
}

// This block defines the configuration for connecting to a syslog server for
// logging.
syslog {
  // This enables syslog logging. Specifying any other option also enables
  // syslog logging.
  enabled = true

  // This is the name of the syslog facility to log to.
  facility = "LOCAL5"
}

// This block defines the configuration for de-duplication mode. Please see the
// de-duplication mode documentation later in the README for more information
// on how de-duplication mode operates.
deduplicate {
  // This enables de-duplication mode. Specifying any other options also enables
  // de-duplication mode.
  enabled = true

  // This is the prefix to the path in Consul's KV store where de-duplication
  // templates will be pre-rendered and stored.
  prefix = "consul-template/dedup/"
}

// This block defines the configuration for exec mode. Please see the exec mode
// documentation at the bottom of this README for more information on how exec
// mode operates and the caveats of this mode.
exec {
  // This is the command to exec as a child process. There can be only one
  // command per Consul Template process.
  command = "/usr/bin/app"

  // This is a random splay to wait before killing the command. The default
  // value is 0 (no wait), but large clusters should consider setting a splay
  // value to prevent all child processes from reloading at the same time when
  // data changes occur. When this value is set to non-zero, Consul Template
  // will wait a random period of time up to the splay value before reloading
  // or killing the child process. This can be used to prevent the thundering
  // herd problem on applications that do not gracefully reload.
  splay = "5s"

  env {
    // This specifies if the child process should not inherit the parent
    // process's environment. By default, the child will have full access to the
    // environment variables of the parent. Setting this to true will send only
    // the values specified in `custom_env` to the child process.
    pristine = false

    // This specifies additional custom environment variables in the form shown
    // below to inject into the child's runtime environment. If a custom
    // environment variable shares its name with a system environment variable,
    // the custom environment variable takes precedence. Even if pristine,
    // whitelist, or blacklist is specified, all values in this option
    // are given to the child process.
    custom = ["PATH=$PATH:/etc/myapp/bin"]

    // This specifies a list of environment variables to exclusively include in
    // the list of environment variables exposed to the child process. If
    // specified, only those environment variables matching the given patterns
    // are exposed to the child process. These strings are matched using Go's
    // glob function, so wildcards are permitted.
    whitelist = ["CONSUL_*"]

    // This specifies a list of environment variables to exclusively prohibit in
    // the list of environment variables exposed to the child process. If
    // specified, any environment variables matching the given patterns will not
    // be exposed to the child process, even if they are whitelisted. The values
    // in this option take precedence over the values in the whitelist.
    // These strings are matched using Go's glob function, so wildcards are
    // permitted.
    blacklist = ["VAULT_*"]
  }

  // This defines the signal that will be sent to the child process when a
  // change occurs in a watched template. The signal will only be sent after
  // the process is started, and the process will only be started after all
  // dependent templates have been rendered at least once. The default value
  // is "" (empty or nil), which tells Consul Template to restart the child
  // process instead of sending it a signal. This is useful for legacy
  // applications or applications that cannot properly reload their
  // configuration without a full reload.
  reload_signal = "SIGUSR1"

  // This defines the signal sent to the child process when Consul Template is
  // gracefully shutting down. The application should begin a graceful cleanup.
  // If the application does not terminate before the `kill_timeout`, it will
  // be terminated (effectively "kill -9"). The default value is "SIGTERM".
  kill_signal = "SIGINT"

  // This defines the amount of time to wait for the child process to gracefully
  // terminate when Consul Template exits. After this specified time, the child
  // process will be force-killed (effectively "kill -9"). The default value is
  // "30s".
  kill_timeout = "2s"
}

// This block defines the configuration for a template. Unlike other blocks,
// this block may be specified multiple times to configure multiple templates.
// It is also possible to configure templates via the CLI directly.
template {
  // This is the source file on disk to use as the input template. This is often
  // called the "Consul Template template". This option is required if not using
  // the `contents` option.
  source = "/path/on/disk/to/template.ctmpl"

  // This is the destination path on disk where the source template will render.
  // If the parent directories do not exist, Consul Template will attempt to
  // create them.
  destination = "/path/on/disk/where/template/will/render.txt"

  // This option allows embedding the contents of a template in the configuration
  // file rather then supplying the `source` path to the template file. This is
  // useful for short templates. This option is mutually exclusive with the
  // `source` option.
  contents = "{{ keyOrDefault \"service/redis/maxconns@east-aws\" \"5\" }}"

  // This is the optional command to run when the template is rendered. The
  // command will only run if the resulting template changes. The command must
  // return within 30s (configurable), and it must have a successful exit code.
  // Consul Template is not a replacement for a process monitor or init system.
  command = "restart service foo"

  // This is the maximum amount of time to wait for the optional command to
  // return. Default is 30s.
  command_timeout = "60s"

  // This is the permission to render the file. If this option is left
  // unspecified, Consul Template will attempt to match the permissions of the
  // file that already exists at the destination path. If no file exists at that
  // path, the permissions are 0644.
  perms = 0600

  // This option backs up the previously rendered template at the destination
  // path before writing a new one. It keeps exactly one backup. This option is
  // useful for preventing accidental changes to the data without having a
  // rollback strategy.
  backup = true

  // These are the delimiters to use in the template. The default is "{{" and
  // "}}", but for some templates, it may be easier to use a different delimiter
  // that does not conflict with the output file itself.
  left_delimiter  = "{{"
  right_delimiter = "}}"

  // This is the `minimum(:maximum)` to wait before rendering a new template to
  // disk and triggering a command, separated by a colon (`:`). If the optional
  // maximum value is omitted, it is assumed to be 4x the required minimum value.
  // This is a numeric time with a unit suffix ("5s"). There is no default value.
  // The wait value for a template takes precedence over any globally-configured
  // wait.
  wait {
    min = "2s"
    max = "10s"
  }
}

Note: Not all fields are required. For example, if you are not using Vault secrets, you do not need to specify a vault configuration. Similarly, if you are not logging to syslog, you do not need to specify a syslog configuration.

For additional security, token may also be read from the environment using the CONSUL_TOKEN or VAULT_TOKEN environment variables respectively. It is highly recommended that you do not put your tokens in plain-text in a configuration file.

Query the nyc3 demo Consul instance, rendering the template on disk at /tmp/template.ctmpl to /tmp/result, running Consul Template as a service until stopped:

consul = "nyc3.demo.consul.io"

template {
  source      = "/tmp/template.ctmpl"
  destination = "/tmp/result"
}

If a directory is given instead of a file, all files in the directory (recursively) will be merged in lexical order. So if multiple files declare a "consul" key for instance, the last one will be used. Please note that symbolic links are not followed.

Commands specified on the command line take precedence over those defined in a config file!

Templating Language

Consul Template consumes template files in the [Go Template][] format. If you are not familiar with the syntax, we recommend reading the documentation, but it is similar in appearance to Mustache, Handlebars, or Liquid.

In addition to the [Go-provided template functions][Go Template], Consul Template exposes the following functions:

API Functions
datacenters

Query Consul for all data centers in the catalog. Data centers are queried using the following syntax:

{{datacenters}}
file

Read and output the contents of a local file on disk. If the file cannot be read, an error will occur. Files are read using the following syntax:

{{file "/path/to/local/file"}}

This example will out the entire contents of the file at /path/to/local/file into the template. Note: this does not process nested templates.

key

Query Consul for the value at the given key. If the key cannot be converted to a string-like value, an error will occur. If the key does not exist, Consul Template will block until the key is present. To avoid blocking, see keyOrDefault or keyExists. Keys are queried using the following syntax:

{{key "service/redis/maxconns@east-aws"}}

The example above is querying Consul for the service/redis/maxconns in the east-aws data center. If you omit the data center attribute, the local Consul data center will be queried:

{{key "service/redis/maxconns"}}

The beauty of Consul is that the key-value structure is entirely up to you!

keyExists

Query Consul for the key. If the key exists, this function will return true, false otherwise. This function does not block if the key does not exist. This is useful for controlling flow:

{{if keyExists "app/beta_active"}}
  # ...
{{else}}
  # ...
{{end}}
keyOrDefault

Query Consul for the value at the given key. If no key exists at the given path, the default value will be used instead. Unlike key, this function will not block if the key does not exist. The existing constraints and usage for keys apply:

{{keyOrDefault "service/redis/maxconns@east-aws" "5"}}

Please note that Consul Template uses a multi-phase evaluation. During the first phase of evaluation, Consul Template will have no data from Consul and thus will always fall back to the default value. Subsequent reads from Consul will pull in the real value from Consul (if the key exists) on the next template pass. This is important because it means that Consul Template will never "block" the rendering of a template due to a missing key from a keyOrDefault. Even if the key exists, if Consul has not yet returned data for the key, the default value will be used instead.

ls

Query Consul for all top-level key-value pairs at the given prefix. If any of the values cannot be converted to a string-like value, an error will occur:

{{range ls "service/redis@east-aws"}}
{{.Key}} {{.Value}}{{end}}

If the Consul instance had the correct structure at service/redis in the east-aws data center, the resulting template could look like:

minconns 2
maxconns 12

If you omit the data center attribute on ls, the local Consul data center will be queried.

node

Query Consul for a single node in the catalog.

{{node "node1"}}

When called without any arguments then the node for the current agent is returned.

{{node}}

You can specify an optional parameter to the nodes call to specify the data center:

{{node "node1" "@east-aws"}}

If the node specified is not found then nil is returned. If the node is found then you are provided with information about the node and a list of services that it provides.

{{with node}}{{.Node.Node}} ({{.Node.Address}}){{range .Services}}
  {{.Service}} {{.Port}} ({{.Tags | join ","}}){{end}}
{{end}}
nodes

Query Consul for all nodes in the catalog. Nodes are queried using the following syntax:

{{nodes}}

This example will query Consul's default data center. You can specify an optional parameter to the nodes call to specify the data center:

{{nodes "@east-aws"}}

This will query Consul for all nodes in the east-aws data center.

secret

Query Vault for the secret data at the given path. If the path does not exist or if the configured Vault token does not have permission to read the path, an error will be returned. If the path exists, but the key does not exist, <no value> will be returned.

{{with secret "secret/passwords"}}{{.Data.password}}{{end}}

The following fields are available:

  • LeaseID - the unique lease identifier
  • LeaseDuration - the number of seconds the lease is valid
  • Renewable - if the secret is renewable
  • Data - the raw data - this is a map[string]interface{}, so it can be queried using Go's templating "dot notation"

If the map key has dots "." in it, you need to access the value using the index function:

{{index .Data "my.key.with.dots"}}

If additional arguments are passed to the function, then the operation is assumed to be a write operation instead of a read operation. The write operation must return data in order to be valid. This is especially useful for the PKI secret backend, for example.

{{ with secret "pki/issue/my-domain-dot-com" "common_name=foo.example.com" }}
{{ .Data.certificate }}
{{ end }}

The parameters must be key=value pairs, and each pair must be its own argument to the function:

{{ secret "path/" "a=b" "c=d" "e=f" }}

Please always consider the security implications of having the contents of a secret in plain-text on disk. If an attacker is able to get access to the file, they will have access to plain-text secrets.

Please note that Vault does not support blocking queries. As a result, Consul Template will not immediatel

Tag summary

Content type

Image

Digest

Size

77.2 MB

Last updated

over 9 years ago

docker pull maauso/docker-consul-template