This is the first release of "Usernetes Generation 2" (#287)

Usernetes: Kubernetes without the root privileges (Generation 2)

Usernetes (Gen2) deploys a Kubernetes cluster inside Rootless Docker,
so as to mitigate potential container-breakout vulnerabilities.

Usernetes (Gen2) is similar to Rootless kind and Rootless minikube,
but Usernetes (Gen 2) supports creating a cluster with multiple hosts.

Components

• Cluster configuration: kubeadm
• CRI: containerd
• OCI: runc
• CNI: Flannel

Requirements

• Rootless Docker:

curl -o install.sh -fsSL https://get.docker.com
sudo sh install.sh
dockerd-rootless-setuptool.sh install

• systemd lingering:

sudo loginctl enable-linger $(whoami)

• cgroup v2 delegation:

sudo mkdir -p /etc/systemd/system/user@.service.d

cat <<EOF | sudo tee /etc/systemd/system/user@.service.d/delegate.conf
[Service]
Delegate=cpu cpuset io memory pids
EOF

sudo systemctl daemon-reload

• Kernel modules:

sudo modprobe vxlan

Usage

See make help.

# Bootstrap a cluster
make up
make kubeadm-init
make install-flannel

# Enable kubectl
make kubeconfig
export KUBECONFIG=$(pwd)/kubeconfig
kubectl get pods -A

# Multi-host
make join-command
scp join-command another-host:~/usernetes
ssh another-host make -C ~/usernetes up kubeadm-join

# Debug
make logs
make shell
make down-v
kubectl taint nodes --all node-role.kubernetes.io/control-plane-

Limitations

• Node ports cannot be exposed automatically. Edit docker-compose.yaml for exposing additional node ports.
• Most of host files are not visible with hostPath mounts. Edit docker-compose.yaml for mounting additional files.
• Some volume drivers such as nfs do not work.

Advanced topics

• Although Usernetes (Gen2) is designed to be used with Rootless Docker, it should work with the regular "rootful" Docker too.
  This might be useful for some people who are looking for "multi-host" version of kind and minikube. (Support for "rootful" is currently broken: #297)