retyc/retycfs-k8s-csi repository overview

⁠RetycFS CSI Driver

RetycFS CSI driver is a CSI driver that uses your Retyc⁠ account to support dynamic provisioning of ReadWriteMany Kubernetes Persistent Volumes via Persistent Volume Claims. Each Persistent Volume is a Retyc dataroom, mounted on the nodes with RetycFS, the filesystem of the retyc CLI⁠, and encrypted end to end on the node before anything reaches the Retyc servers.

Each volume mounted on a node runs in its own mount pod, so restarting or upgrading the driver cuts no mount. Volumes can be expanded, and their versioning switched through a VolumeAttributesClass.

⁠Requirements

• Kubernetes 1.34 or later;
• nodes that can run RetycFS (see the mount pods⁠) and reach api.retyc.com and auth.retyc.com;
• a namespace for the driver that allows privileged pods (Pod Security privileged): retycfs-system, created below;
• a Retyc organization, its UUID, and an administrator of it who approves the driver's machine account. The driver holds no token and no passphrase: it enrolls a machine account⁠.
• for a custom mount.image.tag or driver image: retyc-cli v1.6.0 or later (machine accounts, retyc machine enroll). The chart's defaults already use it.

⁠How to deploy RetycFS CSI driver to your cluster

⁠With Helm

Follow the instructions from the helm chart README⁠.

The tl;dr is

$ kubectl create namespace retycfs-system
$ kubectl label namespace retycfs-system pod-security.kubernetes.io/enforce=privileged
$ helm repo add retyc https://retyc.github.io/retycfs-k8s-csi/
$ helm install retycfs-csi retyc/retycfs-csi --namespace retycfs-system \
    --set identity.organizationID=<Retyc organization UUID>

The enroller requests a machine account for the cluster; helm install does not wait for it. The pods are ready at once, and the claims of the cluster-wide classes stay Pending until an administrator of the organization approves it, in Retyc (Organization > Machines) after comparing the fingerprint, or with the CLI:

$ kubectl -n retycfs-system get retycidentity default -o wide      # PHASE Pending, FINGERPRINT XXXX-...
$ retyc machine requests
$ retyc machine approve <enrollment-id> --fingerprint XXXX-...

Whether approved or not, the controller pod must be 3/3 Running, the node pods 2/2 and the enroller 1/1 (see doc/troubleshooting.md⁠):

$ kubectl -n retycfs-system get pods -l app.kubernetes.io/instance=retycfs-csi

Without identity.organizationID, no cluster-wide identity exists and only the per-namespace ones are served (below).

⁠Test your environment

Deploy the test resources, a claim with a writer pod and a reader pod:

$ kubectl create -f https://raw.githubusercontent.com/retyc/retycfs-k8s-csi/master/examples/manifests/rwx-test.yaml
$ kubectl logs -f retycfs-reader

The reader prints the lines the writer appends. Check retyc dataroom ls, or the Retyc web app: the dataroom is there, named after the volume. The volume's mount pod runs next to the driver:

$ kubectl -n retycfs-system get pods -l app.kubernetes.io/name=retycfs-mount

Delete the test resources:

$ kubectl delete -f https://raw.githubusercontent.com/retyc/retycfs-k8s-csi/master/examples/manifests/rwx-test.yaml

The dataroom is gone as well.

⁠Deploying your own PersistentVolumeClaims

Every claim needs the csi.retycfs.com/uid annotation: every file of the volume belongs to that UID, and only pods running as that UID can write (others read). Use accessModes: [ReadWriteMany] and one of the storage classes:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: shared-docs
  annotations:
    csi.retycfs.com/uid: "1000"
spec:
  accessModes: [ReadWriteMany]
  storageClassName: retycfs-rwx
  resources:
    requests:
      storage: 20Gi

• retycfs-rwx: deleting the claim deletes the dataroom;
• retycfs-rwx-retain: deleting the claim keeps the dataroom;
• retycfs-rwx-tenant: same, with the machine account of the claim's namespace instead of the cluster-wide one. The namespace applies a RetycIdentity named default and has it approved first (see doc/configuration.md⁠). retycfs-rwx and retycfs-rwx-retain exist only with identity.organizationID.

The size requested is reserved on your plan, 20 GiB at least. Run the pods with securityContext.runAsUser set to the annotation's UID, and mount the volume with mountPropagation: HostToContainer, as in the examples: when a mount pod dies and is recreated, the pod then sees its volume come back without restarting (see doc/architecture.md⁠). The other annotations (cache, resources of the mount pod, read-only...) are in doc/configuration.md⁠.

A dataroom belongs to the machine account that created it. To add people to it at creation, use the csi.retycfs.com/initial-members annotation, and read Dataroom lifecycle⁠ before retiring an identity.

Every pod mounting the claim, on any node, shares the same dataroom. It behaves like a network drive: close returns once the file is stored, a change made on one node shows up on the others within the cache TTL (one minute by default), and concurrent writes to one file are not safe. Good for documents, configuration and artefacts; not for databases.

⁠Documentation

• charts/retycfs-csi/README.md⁠: chart values
• doc/configuration.md⁠: PVC annotations, static PVs, versioning, identities, dataroom ownership transfer, Redis cache, metrics
• doc/troubleshooting.md⁠: mount pod logs, exit codes, events
• doc/architecture.md⁠: how it works, mount pods, recovery, consistency, security, limitations
• doc/testing.md⁠: development environment and tests

⁠Development

$ make build test lint     # driver
$ make helm-lint           # chart
$ make image               # container image
$ cd vm && tofu apply     # k3s in Incus VMs (vm/.env: dev/prod chart, multi/single node), then: make image-dev vm-load vm-helm

⁠License

MIT⁠ - © Retyc / TripleStack SAS