What is Konky?
Konky (Kubernetes + chaos monkey) is a randomized event scheduler. It uses the Docker API to randomly 'stop' containers belonging to a specified Kubernetes Pod.
A randomly selected container in the Pod will be the target of the "failure". Konky uses a container label set by Kubernetes to restrict possible targets to only members of that Pod. For instance, nginx containers in a Pod using the default namespace may have the key-value label: "io.kubernetes.pod.name": "default/nginx".
The "failures" will occur periodically based upon a randomly computed delay from Konky's launch time, and thereafter from the most recent container "failure" time.
By default, Konky uses Docker's Unix domain socket to select running containers and issue 'stop' requests. It must run on the same host as the target Pod and have appropriate privileges to access this socket. Alternatively, Konky can run on a different network host and access an unsecured Docker Remote API endpoint corresponding to the target Pod.
All runtime configuration on Konky occurs through environment variables.
All Konky actions are logged to STDOUT or STDERR. The output is available through 'docker logs'.
What is this used for?
Konky is used to verify the fault tolerance of cloud applications. For instance, it ensures that the Kubernetes kubelet node agent is detecting failed containers and launching replacements as required. Konky was designed so that is can be easily used with other application cluster managers which support Docker containers.
How was the image built?
Konky is coded in Go. The Dockerfile is simply:
FROM scratch COPY konky / CMD ["/konky"]
How can I run it?
Konky can be launched with (for example):
$ sudo docker run \ --detach=true \ --volume=/var/run/docker.sock:/var/run/docker.sock \ --env=KONKY_TARGETPODNAME=nginx \ --name=konky \ bahadley/konky
Configurable environment variables
Konky uses default values for most of its environment variables. Only KONKY_TARGETPODNAME must be manually set for every execution.
KONKY_PODKEY=io.kubernetes.pod.name (Docker container label key. The key-value pairs used as Docker container labels can be viewed with 'docker inspect'.)
KONKY_TARGETPODNAME=[REQUIRED] (Docker container label value. Konky automatically adds the prefix 'default/' if KONKY_PODKEY is set to the default value and a Kubernetes namespace isn't already provided. For example, the above 'docker run' results in 'default/nginx' for the label matching value. Konky uses prefix matching for the Pod name. In this case, any Pod name matching nginx* will be a target candidate.)
KONKY_RANDOMRANGELB=30 (Inclusive lower bound in seconds for the random delay range)
KONKY_RANDOMRANGEUB=120 (Exclusive upper bound in seconds for the random delay range)
KONKY_UNIXSOCKET=/var/run/docker.sock (Docker Unix domain socket)
KONKY_USEREMOTEADDR=false [true|false] (Use an unsecured Docker Remote API endpoint instead of the Unix domain socket.)
KONKY_REMOTEADDR=172.17.42.1:2375 (IP and port of unsecured Docker Remote API endpoint. The default value uses the commonly occurring gateway address for the default Docker bridge. This will be convenient if Konky targets containers on the local host and the Docker daemon listens for Remote API requests on all interfaces (ie. -H tcp://0.0.0.0:2375). Consider running Konky with the '--net="host"' option if the daemon only listens on the 'localhost' interface.)
KONKY_DRYRUN=false [true|false] (Don't 'stop' a container, but log target selections)
KONKY_DEBUG=false [true|false] (Log extra debug information)
Can Konky be deployed as a Kubernetes Pod?
Yes, Konky can use a Docker Remote API endpoint when it is deployed as a Pod. The following shows a basic configuration using YAML:
apiVersion: v1 kind: Pod metadata: name: konky labels: app: konky spec: containers: - image: bahadley/konky name: konky env: - name: "KONKY_TARGETPODNAME" value: "nginx" - name: "KONKY_USEREMOTEADDR" value: "true"