Announcements

We are looking for maintainers, reach out in #5432.

Deprecation / Removal

• Remove contrib/vault (Outdated since 2018) (#7400)
• Drop support for calico version 3.15.x (#7545)

Major changes

• Replace inventory group kube-master with kube_control_plane (#7256) (see Notes 5)
• Move kubernetes/master to kubernetes/control-plane (#7218) (see Notes 1)
• Move recover_control_plane/master to control-plane (#7236) (see Notes 2)
• Replace KUBE_MASTERS with KUBE_CONTROL_HOSTS (#7257) (see Notes 3)
• Rename ansible groups to use _ instead of - (#7552) (see Notes 7)
• Add AlmaLinux support (#7538)
• Add terraform support for Exoscale (#7141)
• Add terraform support for Vsphere (#7306)
• Add terraform support for UpCloud (#7360)
• Support for CentOS 8 and derivatives is considered stable (#7615)
• Support dual stack IPv4 & IPv6 networking (#6859)
• Auto renew control plane certificates (#7358) (see Notes 4)
• Add auto_renew_certificates_systemd_calendar to configure when K8S certificates renewal runs (#7490)
• Specify runAsGroup, allow safe sysctls by default (#7399)
• Add KubeSchedulerConfiguration for k8s 1.19 and up (#7351) (see Notes 6)
• Add script for generate download files and images list (#7561)
• Terraform 0.12+ is now required to run scripts under contrib/terraform/aws (#7576)
• Allow using ansible 2.10.x to deploy Kubespray (#7600)
• Add a contrib playbook (os-manage) to disable service firewall for Kubespray development and test (#7431)

Applications

• [Krew] Add krew support (#7464)
• [Openstack] Make sure worker rules is applied on workers (#7279)
• [Openstack] Write openstack controller manifests with correct perms (#7284)
• [Openstack] Allow users to set image_uuid instead of name, this allows the use of openstack community images (#7283)
• [Openstack] Use image id instad of name (#7293)
• [Openstack] Update Cinder CSI driver to v1.20.0 (#7280)
• [Openstack] Add most_recent = true while retrieving the latest image (#7376)
• [Openstack] Add external_openstack_enable_ingress_hostname option for external-openstack-cloud-controller-manager (#7572)
• [Metallb] Introduces optional tolerations and nodeSelector for metallb components (controller and speaker) (#7334)
• [CSI] Add suport of Vsphere CSI driver 2.X versions (#7480)
• [External-Provisioner] Add new variable "local_volume_provisioner_use_node_name_only" to configure local volume provisioner "useNodeNameOnly" option (#7421)

Container managers

• [CRI-O] Add experimental cri-o support for Amazon Linux 2 (#7353)
• [CRI-O] Add support for configuring cri-o pids_limit (#7525)
• [CRI-O] Fix support for cri-o on OracleLinux and add support for AlmaLinux (#7541)
• [Containerd] Fix reset.yml failing when using containerd (#7308)
• [Containerd] Add privileged_without_host_devices support (#7343)
• [Containerd] Update config.toml to V2 and set default runtime to io.containerd.runc.v2 and cgroup to systemd (#7398)
• [Containerd] Add containerd_extra_args (#7461)
• [Containerd] Add nerdctl cli tool for containerd users (#7500)
• [Containerd] Add support for Amazon Linux 2(#7595)
• [Docker] docker_dns_servers_strict had different default values, the default is now the same everywhere: false (#7499)
• [Docker] Add enablerepo: amzn2extra-docker to allow docker installation on Amazon linux (#7507)
• [crun] Update and changed the default crun version to v0.19 (#7433)
• [crictl] Change the owner of /etc/crictl.yaml to root (#7254)

Network

• [Calico] Fixup check when ipipMode / vxlanMode is not present (#7195)
• [Calico] Support for dual stack (IPv4 & IPv6) network deployment using Calico is introduced as an opt-in feature (#6859)
• [Calico] Add option to use calico with azure when using calico in vxlan (#7300)
• [Calico] Download Calico KDD CRDs (#7372)
• [Calico] Add the ability to customize calico's bird port, via calico_bird_listen_port variable (#7419)
• [Calico] Add new variable calico_node_startup_loglevel to configure CALICO_STARTUP_LOGLEVEL (Default to error) (#7530)
• [Calico] Allow specifying overriding BGP peer name (#7591)
• [Calico] Enables Calico serviceAccount token monitoring and update of /etc/cni/net.d/calico-kubeconfig if need be (#7586)
• [Calico] Add support to advertise MetalLB allocated IPs through Calico when using Calico 3.18 and greater (#7593)
• [Cilium] Allow cilium to be deployed with transparent encryption (#7342)
• [Cilium] Add cilium_ipam_mode variable (#7418)
• [Cilium] Move cilium kvstore settings to configmap (#7462)
• [Cilium] Update Cilium documentation and overall update of cilium role (#7521)
• [Ambassador] Add ingress_ambassador_multi_namespace setting, allows Ambassador operator to watch all namespaces for AmbassadorInstallation CRD resources (#7516)
• [Flannel] Add image_arch in image tag (#7560)

Other note worthy changes

• Added the ping_access_ip variable to enable(default)/disable ping test during preinstall (#7020)
• Rework proxy support (#7095)
• Remove ignore_errors from drain tasks and enable retires (#7151)
• Add other masters sequentially, not in parallel (#7166)
• Add 2 variables for upgrade, to prompt (upgrade_node_confirm, default false) and delay (upgrade_node_pause_seconds, default 0 seconds) (#7168)
• Change node-role.kubernetes.io from master to control-plane (#7183)
• Add retries to drain during upgrade. Allow leaving nodes cordoned after drain failure. Allow continuing upgrade if drain fails (#7227)
• Vagrantfile: always recreate inventory symlink (#7245)
• Updated etcd cert check tasks to detect when new cert gen is required (#7219)
• Only use stat get_checksum: yes when needed (#7270)
• Match on os-release ID / VARIANT_ID (#7269)
• Fix issue with kubeadm when *_PROXY variables are present in the environment (#7275)
• Kubespray now ignores *_PROXY vars found in your environment and only uses proxy configuration from the inventory (#7309)
• Facts.yaml: reduce the number of setup calls by ~7x (#7286)
• Fixup kubelet.conf to point to kubelet-client-current.pem (#7347)
• Check for dummy kernel module (#7348)
• Disable gather_facts for correctly work via bastion (#7265)
• Add etcd max snapshot and wals (#7382)
• Add cryptography module installation (#7404)
• Allow connecting to bastion via non-standard SSH port (#7396)
• Remove local lb privileged securityContext (#7437)
• Regenerate apiserver.crt on all controle-plane nodes when needed instead of just the first one (#7463)
• Check if python netaddr is installed and if Jinja is recent enough (#7486)
• Add ingress controller ingress-class var (#7522)
• Update Dockerfile to reduce Kubespray image size (#7556)
• Change kubeadm coredns addon images name to coredns/coredns (#7570)
• Allow usage of jinja2_native=True (#7612 / #7606)

Component versions:

• Kubernetes v1.20.7
• Etcd 3.4.13
• Docker 19.03
• Containerd 1.4.4
• CRI-O 1.20
• CNI-plugins v0.9.1
• Calico v3.17.4
• Cilium 1.8.9
• Flannel 0.13.0
• Kube-Router 1.2.2
• Multus 3.7
• Kube-ovn 1.6.2
• Weave 2.8.1
• CoreDNS 1.7.0
• Nodelocaldns 1.17.1
• Helm 3.5.4
• Nginx-ingress 0.43.0
• Cert-manager 1.0.4
• Kubernetes Dashboard v2.2.0

Known issues

• Ansible 2.11 is not supported and using it will results in errors
• Using Docker container engine could prompt "PLEG IS NOT HEALTHY" error, due to a runc bug, please see this issue for more information.

Notes

1. The role kubernetes/master has been renamed to kubernetes/control-plane, if using the role kubernetes/master solely on previous Kubespray, it is necessary to update the specified role.
2. The role recover_control_plane/master has been renamed to recover_control_plane/control-plane. If using the role recover_control_plane/master solely on previous Kubespray, it is necessary to update the specified role.
3. inventory_builder starts referring the environment variable KUBE_CONTROL_HOSTS to get the number of control-plane nodes, it still refers KUBE_MASTERS but it will be not referred after some deprecation cycles. Please specify KUBE_CONTROL_HOSTS if now specifying KUBE_MASTERS
4. You can enable control plane certificates automatic renewal using auto_renew_certificates, or manually use k8s-certs-renew.sh force_certificate_regeneration is removed as it was only renewing the api server certs and not all the other ones
5. The inventory group kube-master has been renamed to kube_control_plane. Please update your inventory file by replacing kube-master if continuing to use the existing inventory file.
6. New vars for configuring kube-scheduler were introduced (including extenders and profiles). Default vaules can be found at roles/kubernetes/control-plane/defaults/main/kube-scheduler.yml
7. Ansible groups were updated to be more consistent with dynamic inventory plugins: k8s-cluster -> k8s_cluster / kube-node -> kube_node / calico-rr -> calico_rr / no-floating -> no_floating