maru/nkube repository overview

⁠nkube (Nested Kubernetes)

nkube is a tool for deploying multinode Kubernetes⁠ clusters on Kubernetes itself. It uses helm⁠ to deploy a chart⁠ consisting of containers running systemd⁠ and docker-in-docker⁠. kubeadm⁠ is then invoked to bootstrap a new Kubernetes cluster.

⁠Usage

While nkube can potentially target any kubernetes deployment, it is currently only tested with minikube. To get started:

• Install minikube⁠
• Install kubectl⁠
• Install Helm⁠
• Start a new minikube cluster:

minikube start

• Initialize helm:

helm init

• Ensure that the ip6_tables module is loaded on the docker host (required for calico):

minikube ssh
sudo modprobe ip6_tables
exit

• From the root of a clone of this repo, start a new nested cluster with the calico plugin. Deployment is likely to take 3-5m, depending on the speed of the host and its network connection:

./start.sh [helm install args]

• Once start.sh has finished, a context will have been added that will allow access to the cluster:

kubectl --context=[cluster-id]

• More than one nested cluster can be deployed at once.

• Since the cluster is deployed with helm, helm commands can be used to manage the cluster (e.g helm delete [cluster id] removes the cluster).

• ssh access to the nodes of the cluster is not supported. Instead, use kubectl exec to gain shell access to the master and node pods.

• The number of nodes can be scaled by setting the replica count of the node deployment. The number of nodes is limited only by the capacity of the hosting cluster.

⁠Warnings

• The use of persistent storage for etcd is currently unsupported. If the nested master fails, the cluster state is lost.
• Due to the way docker-in-docker handles volumes, manual cleanup on the host docker is required:

docker volume ls -qf dangling=true | xargs -r docker volume rm