Public Repository

Last pushed: 7 months ago
Short Description
A custom hyperkube build that includes an admission controller that queries OPA for decisions.
Full Description

OPA Admission Controller

Overview

This kube-apiserver build includes a custom admission controller that queries OPA to obtain policy decisions.

To use this kube-apiserver:

  1. Run the kube-apiserver with the "OPA" admisson controller enabled and configured.
  2. Deploy OPA on top of Kubernetes.
  3. Expose OPA's API as a Kubernetes Service.
  4. Annotate the OPA Kubernetes Service to control fail-open vs. fail-closed behavior.

For more information about deploying OPA on top of Kubernetes, loading data and policies into OPA, and so on, go to openpolicyagent.org.

Policies

The policy you write for OPA to control admission must return a JSON object with the following fields:

{
    "errors": <array>
    "annotations": <object>
}
  • The errors field is an array of strings indicating whether to allow the request or not. If the array is non-empty, the request will be denied.

  • The annotations field is an object value containing desired values for annotations. The admission controller will set the annotation fields on the resource to the desired values. If there are existing annotations in the resource with the same key, they are replaced. This field is optional.

In the future, the admission controller may be extended to support additional or arbitrary resource attributes.

The simplest policy admits all resources all the time and does not set any annotations:

package example.admission

allow = true

errors["request denied by administrative policy"] {
  not allow
}

You are free to choose any package name you like. Just be sure to make the
admission controller configuration match (see Config for details).

In this case, the admission controller would query OPA:

POST /v1/data/example/admission HTTP/1.1

And receive the following response:

HTTP/1.1 200 OK
Content-Type: application/json

{
  "result": {
    "allow": true,
    "errors": []
  }
}

Because "allow" is always true, "errors" is always empty. As a result, the admission controller will admit all requests.

If we flip the "allow" rule to false, then kubectl commands to create and update resources will fail:

package example.admission

allow = false

errors["request denied by administrative policy"] {
  not allow
}

Example:

kubectl create -f nginx.yaml

Output:

Error from server (Forbidden): error when creating "nginx.yaml": replicasets.extensions "nginx" is forbidden: reason(s): request denied by administrative policy

Notice that the kubectl output includes the error message we defined in the policy. If there are multiple errors they will all be included.

For more information on how to write policies, go to openpolicyagent.org.

The rest of this document describes the steps to configure and run the admission controller.

Configuring and Running the Admission Controller

To enable the admission controller, include "OPA" in the list passed to the kube-apiserver's --admission-control argument. If you are not specifying --admission-control already, run kube-apiserver --help to get the default value and then append "OPA" to the end. Then start kube-apiserver with the new value.

In addition, you must supply an admission control config file via the --admission-control-config-file <path> argument. See below for details.

Once Kubernetes is running, you can deploy OPA on top. Then, create a Service to expose the OPA API. The admission controller uses the Service to discover OPA. The following Service types are supported: ClusterIP, NodePort, and ExternalName.

The following limitations currently apply to the OPA deployment and the admission controller:

  • If you are using a replicated resource type (e.g., a Deployment or ReplicaSet) to deploy OPA then the number of replicas should be set to 1.

  • The admission controller only supports HTTP. Do not configure OPA to expose its API over HTTPS.

  • The admission controller does not support the LoadBalancer Service type. You will receive an error from the admission controller if you try to expose the OPA API using a LoadBalancer service.

These limitations are implementation details that we will resolve in the near future!

The configuration file should contain the following configuration for the OPA admission controller:

# Top-level key to namespace OPA admission controller config.
# This must be "opa".
opa:
  serviceName: <string>
  serviceNamespace: <string>
  servicePortName: <string>
  path: <string>
  • The serviceName field specifies the name of the Kubernetes Service exposing the OPA API.
  • The serviceNamespace field specifies the namespace of the Kubernetes Service exposing the OPA API. For example: default.
  • The servicePortName field specifies the name of the port on the Kubernetes Service to use. For example: http.
  • The path field specifes the path of the OPA Data API document to query. For example: /v1/data/example/admission would be used for the policy from above. This is relative to the base URL constructed from the Service's IP/hostname and port.

Fail-open vs. Fail-closed

By default, the admission controller fails open. If any failures are encountered before the admission controller interprets the errors field, the request will be admitted.

Administrators can change this behaviour by annotating the OPA Kubernetes Service (which is identified by the serviceName and serviceNamespace config fields) with:

{
    "openpolicyagent.org/fail-closed": "true"
}

If the admission controller reads the openpolicyagent.org/fail-closed and the value is "true" then any failures will cause the admission controller to deny the request. This way, administrators can configure the fail-closed behaviour once OPA has been successfully deployed.

We add an annotation to the OPA Kubernetes Service to provide a signal to clients of OPA (such as the admission controller) that OPA is up and ready for requests. In the case of Kubernetes, this is important as it allows admins to bootstrap Kubernetes with OPA by:

  • First, deploying the Kubernetes apiserver with the admission controller enabled (but failing open).
  • Then deploying OPA on top of the Kubernetes cluster.
  • Then loading the minimum required data & policy into OPA.
  • Finally, toggling the openpolicyagent.org/fail-closed annotation on the OPA Kubernetes Service.
Docker Pull Command
Owner
openpolicyagent

Comments (0)