viaductoss/ksops

By viaductoss

•Updated 5 months ago

KSOPS - A Flexible Kustomize Plugin for SOPS Encrypted Resources

Image

1M+

Overview Tags

viaductoss/ksops repository overview

⁠KSOPS - A Flexible Kustomize Plugin for SOPS Encrypted Resource

Background⁠
Overview⁠
Requirements⁠
Example⁠
Generator Options⁠
Development and Testing⁠
Argo CD Integration 🤖⁠

⁠Background

At Viaduct⁠, we manage our Kubernetes resources via the GitOps⁠ pattern; however, we could not find a solution compatible with our stack for managing secrets via the GitOps paradigm. We built KSOPS to connect kustomize⁠ to SOPS⁠ and integrated it with Argo CD⁠ to safely manage our secrets the same way we manage the rest our Kubernetes manifest.

⁠Overview

KSOPS, or kustomize-SOPS, is a kustomize⁠ plugin for SOPS encrypted resources. KSOPS can be used to decrypt any Kubernetes resource, but is most commonly used to decrypt encrypted Kubernetes Secrets and ConfigMaps. As a kustomize⁠ plugin, KSOPS allows you to manage, build, and apply encrypted manifests the same way you manage the rest of your Kubernetes manifests.

⁠Requirements

Go⁠
kustomize⁠ built with Go (See details below⁠)
SOPS⁠
gpg

⁠Example

⁠1. Download KSOPS

# export GO111MODULE=on
go get -u github.com/viaduct-ai/kustomize-sops
# cd into the root directory
cd $GOPATH/src/github.com/viaduct-ai/kustomize-sops

⁠2. Install (or Reinstall) the Latest kustomize via Go

# KSOPS is built with latest kustomize
# If you want to change versions, update the installation script with your desired version and make sure to check that the KSOPS tests still pass
# If you want to change versions below kustomize v3.3.0, use the KSOPS v1.0 or go-1.12 release!
make kustomize

⁠3. Setup kustomize Plugin Path

# Don't forget to define XDG_CONFIG_HOME in your .bashrc/.zshrc
echo "export XDG_CONFIG_HOME=\$HOME/.config" >> $HOME/.bashrc
source $HOME/.bashrc

⁠4. Build and Install KSOPS Plugin

make install

⁠5. Import Test PGP Keys

To simplify local development and testing, we use PGP test keys. To import the keys, run the following command:

make import-test-keys

If you are following this tutorial, be sure to run this before the following steps. The PGP keys will also be imported when you run make test

See SOPS⁠ for details.

⁠6. Configure SOPS via .sops.yaml

For this example and testing, KSOPS relies on the SOPS creation rules defined in .sops.yaml. To make encrypted secrets more readable, we suggest using the following encryption regex to only encrypt data and stringData values. This leaves non-sensitive fields, like the secret's name, unencrypted and human readable.

Note: You only have to modify .sops.yaml if you want to use your key management service in this example instead of the default PGP key imported in the previous step.

creation_rules:
  - encrypted_regex: '^(data|stringData)$'
    # Specify kms/pgp/etc encryption key
    # This tutorial uses a local PGP key for encryption.
    # DO NOT USE IN PRODUCTION ENV
    pgp: 'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4'
    # Optionally you can configure to use a providers key store
    # kms: XXXXXX
    # gcp_kms: XXXXXX

⁠7. Create a Resource

# Create a local Kubernetes Secret
cat <<EOF > secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
data:
  username: YWRtaW4=
  password: MWYyZDFlMmU2N2Rm
EOF

⁠8. Encrypt the Resources

# Encrypt with SOPS CLI
# Specify SOPS configuration in .sops.yaml
sops -e secret.yaml > secret.enc.yaml

⁠9. Define KSOPS kustomize Generator

# Create a local Kubernetes Secret
cat <<EOF > secret-generator.yaml
apiVersion: viaduct.ai/v1
kind: ksops
metadata:
  # Specify a name
  name: example-secret-generator
files:
  - ./secret.enc.yaml
EOF

⁠10. Create the kustomization.yaml

Read about kustomize plugins⁠

cat <<EOF > kustomization.yaml
generators:
  - ./secret-generator.yaml
EOF

⁠11. Build with kustomize 🔑

# Build with kustomize to verify
kustomize build --enable_alpha_plugins .

⁠Troubleshooting

⁠kustomize Go Plugin Caveats

Detailed example of kustomize Go plugin⁠

⁠Sanity Checks

Validate ksops.so is in the kustomize plugin path
- $XDG_CONFIG_HOME/kustomize/plugin/viaduct.ai/v1/ksops/ksops.so
Check your kustomize executable was built by Go
- which kustomize
- kustomize version
Check the Go version in go.mod matches the Go version used to build kustomize
Check the kustomize version specified in go.mod matches the installed version of kustomize
- kustomize version

⁠Check Existing Issues

Someone might have already encountered your issue.

https://github.com/viaduct-ai/kustomize-sops/issues⁠

⁠Generator Options

KSOPS supports the same annotation based generator options as kustomize exec plugins⁠. The supported annotations are:

kustomize.config.k8s.io/needs-hash
kustomize.config.k8s.io/behavior

For information, read the kustomize exec plugin generator options documenation⁠.

⁠Development and Testing

Before developing or testing KSOPS, ensure all external requirements⁠ are properly installed.

# Setup development environment
make setup

⁠Development

KSOPS implements the kustomize⁠ plugin API in ksops.go.

KSOPS's logic is intentionally simple. Given a list of SOPS encrypted Kubernetes manifests, it iterates over each file and decrypts it via SOPS decrypt⁠ library. KSOPS assumes nothing about the structure of the encrypted resource and relies on kustomize⁠ for manifest validation. KSOPS expects the encryption key to be accessible. This is important to consider when using KSOPS for CI/CD.

⁠Testing

Testing KSOPS requires:

Configuring a encryption key and other SOPS configuration in .sops.yaml
Building KSOPS as a Go plugin
Copying the Go plugin to the kustomize⁠ plugin path
Generating encrypted test files
Running the Go tests

Everything but setting up .sops.yaml is handle for you by make test. After defining .sops.yaml, test KSOPS running the following command from the repo's root directory:

make test

⁠Argo CD Integration 🤖

KSOPS becomes even more powerful when integrated with a CI/CD pipeline. By combining KSOPS with Argo CD⁠, you can manage Kubernetes secrets via the same Git Ops pattern you use to manage the rest of your kubernetes manifests. To integrate KSOPS and Argo CD⁠, you will need to update the Argo CD ConifgMap and create a strategic merge patch⁠ or a custom Argo CD build⁠. Don't forget to inject any necessary credentials (i.e AWS credentials) when deploying the Argo CD⁠ + KSOPS build!

KSOPS Docker Image⁠

⁠Enable Kustomize Plugins via Argo CD ConfigMap

As of now to allow Argo CD⁠ to use kustomize⁠ plugins you must use the enable_alpha_plugins flag. This is configured by the kustomize.buildOptions setting in the Argo CD⁠ ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  labels:
    app.kubernetes.io/name: argocd-cm
    app.kubernetes.io/part-of: argocd
data:
  kustomize.buildOptions: "--enable_alpha_plugins"

⁠KSOPS Repo Sever Patch

The simplest way to integrate KSOPS with Argo CD⁠ is with a strategic merge patch⁠ on the Argo CD repo server deployment. The patch below uses an init container to build KSOPS and kustomize⁠ and volume mount to inject the KSOPS plugin and override the kustomize⁠ executable.

# argo-cd-repo-server-ksops-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: argocd-repo-server
spec:
  template:
    spec:
      # 1. Define an emptyDir volume which will hold the custom binaries
      volumes:
        - name: custom-tools
          emptyDir: {}
      # 2. Use an init container to download/copy custom binaries into the emptyDir
      initContainers:
        - name: install-ksops
          # Match Argo CD Go version
          image: viaductoss/ksops:v2.1.2-go-1.14
          command: ["/bin/sh", "-c"]
          args:
            - echo "Installing KSOPS...";
              export PKG_NAME=ksops;
              mv ${PKG_NAME}.so /custom-tools/;
              mv $GOPATH/bin/kustomize /custom-tools/;
              echo "Done.";
          volumeMounts:
            - mountPath: /custom-tools
              name: custom-tools
      # 3. Volume mount the custom binary to the bin directory (overriding the existing version)
      containers:
        - name: argocd-repo-server
          volumeMounts:
            - mountPath: /usr/local/bin/kustomize
              name: custom-tools
              subPath: kustomize
              # Verify this matches a XDG_CONFIG_HOME=/.config env variable
            - mountPath: /.config/kustomize/plugin/viaduct.ai/v1/ksops/ksops.so
              name: custom-tools
              subPath: ksops.so
          # 4. Set the XDG_CONFIG_HOME env variable to allow kustomize to detect the plugin
          env:
            - name: XDG_CONFIG_HOME
              value: /.config
        ## If you use AWS or GCP KMS, don't forget to include the necessary credentials to decrypt the secrets!
        #  - name: AWS_ACCESS_KEY_ID
        #    valueFrom:
        #      secretKeyRef:
        #        name: argocd-aws-credentials
        #        key: accesskey
        #  - name: AWS_SECRET_ACCESS_KEY
        #    valueFrom:
        #      secretKeyRef:
        #        name: argocd-aws-credentials
        #        key: secretkey

⁠Custom Argo CD w/ KSOPS Dockerfile

Alternatively, for more control and faster pod start times you can build a custom docker image.

ARG ARGO_CD_VERSION="v1.5.5"
# Always match Argo CD Dockerfile's Go version!
# https://github.com/argoproj/argo-cd/blob/master/Dockerfile
ARG KSOPS_VERSION="v2.1.2-go-1.14"

#--------------------------------------------#
#--------Build KSOPS and Kustomize-----------#
#--------------------------------------------#

FROM viaductoss/ksops:$KSOPS_VERSION as ksops-builder

#--------------------------------------------#
#--------Build Custom Argo Image-------------#
#--------------------------------------------#

FROM argoproj/argocd:$ARGO_CD_VERSION

# Switch to root for the ability to perform install
USER root

# Set the kustomize home directory
ENV XDG_CONFIG_HOME=$HOME/.config
ENV KUSTOMIZE_PLUGIN_PATH=$XDG_CONFIG_HOME/kustomize/plugin/

ARG PKG_NAME=ksops

# Override the default kustomize executable with the Go built version
COPY --from=ksops-builder /go/bin/kustomize /usr/local/bin/kustomize

# Copy the plugin to kustomize plugin path
COPY --from=ksops-builder /go/src/github.com/viaduct-ai/kustomize-sops/*  $KUSTOMIZE_PLUGIN_PATH/viaduct.ai/v1/${PKG_NAME}/

# Switch back to non-root user
USER argocd

Tag summary

Recent tags

Content type

Image

Digest

sha256:4def9fdd4…

Size

51.5 MB

Last updated

5 months ago

docker pull viaductoss/ksops