github kubernetes-sigs/kubespray v2.32.0
on GitHub

5 hours ago

Highlights

  • Kubernetes 1.36 is the default minor version
  • Ansible environment moves to Ansible 12 / ansible-core 2.19, with breaking changes to conditionals and templating. See Urgent Upgrade Notes before upgrading.
  • New operating system support includes Ubuntu 26.04, openSUSE Leap 16.0, and openSUSE MicroOS with containerd.

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

  • Ansible environment (action required): Kubespray now requires ansible-core >=2.19.0,<2.20.0 and ships with Ansible 12. Before running playbooks, recreate or update the virtual environment on the Ansible control host using this release's requirements.txt. Ansible 2.19 requires conditional expressions to return booleans; -e key=value supplies strings. Pass boolean overrides as JSON (for example, -e '{"drain_nodes": true, "drain_fallback_enabled": true}') or use unquoted booleans in YAML inventory or extra-vars files (-e @vars.yml). Review custom Jinja templates in inventories and integrated playbooks for compatibility with the templating changes described in the Ansible 12 porting guide. The documentation examples have been updated accordingly. (#13468, @VannTen; #13526, @shashankvarma499)
  • Node Feature Discovery (action required): The default version is now 0.19.0. Remove node_feature_discovery_enable_nodefeature_api, node_feature_discovery_master_crd_controller, and node_feature_discovery_master_instance from inventory overrides. gRPC mode and executable hooks under source.d are no longer supported. Custom NodeFeatureRule templates must no longer use env, expandenv, or getHostByName; review and migrate affected configurations before upgrading. (#13512, @vashian)
  • Pinned Helm versions (action required): Helm now defaults to 3.22.0. Checksums for Helm 3.16.x, 3.17.x, and 3.18.x have been removed. If inventory overrides pin helm_version to one of these versions, remove the override to use the default or select an available version from the retained 3.20.x, 3.21.x, or 3.22.x series. (#13535, @guoard)
  • Cilium Gateway API: Gateway API CRDs now default to 1.6.2, meeting Cilium 1.20's minimum requirement of Gateway API 1.6.1. If you enable Cilium's Gateway API support and pin gateway_api_version below 1.6.1, update or remove that override before upgrading. Gateway API 1.6.0 and 1.6.1 checksums are also available, but 1.6.0 does not meet Cilium 1.20's Gateway API requirement. (#13534, @yankay)
  • etcd metrics discovery: The etcd-metrics Service now uses EndpointSlice instead of Endpoints to avoid duplicate metrics. If you use this Service, update scrapers that still discover Endpoints. With Prometheus Operator, set spec.serviceDiscoveryRole: EndpointSlice on the relevant ServiceMonitor or Prometheus resource, and grant Prometheus permission to get, list, and watch endpointslices in the discovery.k8s.io API group. Verify etcd scrape targets after upgrading. (#12846, @jmeza-xyz)
  • Debian 11 and macvlan: Debian 11 (Bullseye) is no longer supported; migrate to a supported operating system before upgrading. macvlan support on Debian has also been removed because it was not functional on Debian 12 or 13. macvlan is no longer tested in CI. (#13483, @VannTen)
  • Debian 13 with Docker: This combination is no longer supported or tested because Debian 13 removed apt-key, which the Docker repository setup still requires. Use containerd on Debian 13, or remain on an operating system supported by the Docker role. (#13499, @VannTen)
  • Local volume provisioner: Entries in local_volume_provisioner_storage_classes now use the upstream storageClassMap camelCase keys. Update inventory overrides: host_dir -> hostDir, mount_dir -> mountDir, volume_mode -> volumeMode, fs_type -> fsType, and block_cleaner_command -> blockCleanerCommand. (#13504, @VannTen)
  • Cilium image overrides: Rename cilium_hubble_certgen_image_{repo,tag} to cilium_certgen_image_{repo,tag} and cilium_hubble_envoy_image_{repo,tag} to cilium_envoy_image_{repo,tag} in inventory overrides. (#13260, @guoard)
  • Control-plane eviction thresholds: eviction_hard_control_plane has been removed. Configure eviction_hard under group_vars/kube_control_plane/ instead. (#13400, @tophac)

Changes by Kind

Feature

  • Add additional_sysctl_min to raise sysctl values only when they are below a configured minimum. Expand the default minimums for common networking and container-density settings. additional_sysctl retains its exact-value behavior. (#13224, @yankay)
  • Add kubeadm_backup_retention_count to clean up old kubeadm backups and temporary upgrade files under /etc/kubernetes/tmp/. The default, -1, disables cleanup; 0 removes all matching files and directories, and positive values retain that many recent entries of each type. (#13415, @wangsifei99)
  • Add openeuler_repo_baseurl to select an alternative openEuler package mirror. (#13134, @yankay)
  • Add an opt-in admission readiness check before kubeadm upgrades with kubeadm_upgrade_preflight_check_enabled (default: false). Configure retries and delay with kubeadm_upgrade_preflight_check_retries and kubeadm_upgrade_preflight_check_delay. A failed check warns but does not block the upgrade. (#13140, @felipe88alves)
  • Add authentication for custom CNI Helm repositories with custom_cni_chart_repository_username and custom_cni_chart_repository_password. (#13294, @A-Soltani255)
  • Add openSUSE Leap 16.0 support and representative KubeVirt CI coverage. (#13473, @NikKumar811)
  • Add openSUSE MicroOS support for containerd-based clusters, including transactional package installation and reboot handling. Docker and CRI-O are not supported on MicroOS. (#13449, @yankay)
  • Add optional apt package cache cleanup during system upgrades with system_upgrade_autoclean: true on Debian-based systems. (#13327, @Jeroen0494)
  • Add optional Prometheus metrics for the kube-vip static pod with kube_vip_metrics_enabled and kube_vip_metrics_port. (#13229, @isumitsolanki)
  • Add Terraform provisioning for OpenNebula 7.2+ in contrib/terraform/opennebula, including automatic Ansible inventory generation. (#13390, @razvan01)
  • Add Ubuntu 26.04 support. (#13443, @yankay)
  • Enable Flannel's nftables backend for Flannel 0.25.5 or newer when kube_proxy_mode: nftables. The nft utility must be installed on the nodes. (#13291, @rtsui-harmonicinc)
  • Make controller-manager node CIDR allocation configurable with kube_controller_manager_allocate_node_cidrs. (#13239, @modzilla99)
  • Make Kubernetes 1.36 the default minor version. See Component Versions below for the selected patch version. (#13219, @tmurakam)
  • Update Cilium to 1.20.2. (#13515, @guoard)
  • Update DNS autoscaler (cluster-proportional-autoscaler) to 1.10.3. (#13529, @vashian)
  • Update nerdctl to 2.3.5. (#13437, @sandaruwijewardhana)
  • Update the default Flannel version to 0.28.9. (#13436, @dhakshin-raghav)
  • Use CRI-O 1.36 by default with Kubernetes 1.36. See Component Versions below for the selected patch version. (#13244, @tmurakam)

Bug or Regression

  • Add missing RBAC permissions for calico-kube-controllers when using the Kubernetes datastore. (#13375, @taiko19xx)
  • Advertise IPv4 as well as IPv6 service cluster IPs through Calico BGP when calico_advertise_cluster_ips: true. (#13503, @VannTen)
  • Allow facts to be gathered and cached for reachable hosts when another inventory host is unreachable. (#13438, @suii2210)
  • Apply cilium_agent_scrape_port, cilium_operator_scrape_port, and cilium_hubble_scrape_port to the Cilium Helm values so these settings take effect. (#13370, @somaz94)
  • Create /opt/cni/bin with root ownership to fix Cilium 1.19.x CNI installation failures. Reject Cilium configurations with a non-root kube_owner during preflight validation. (#13385, @emma034186-stack)
  • Custom CNI static manifests can set custom_cni_namespace to an empty value to apply resources across multiple namespaces. The default remains kube-system. (#13091, @yankay)
  • Disable ExtendWebSocketsToKubelet when container_manager: docker is used with Kubernetes 1.36 or newer, because cri-dockerd does not support WebSocket streaming. (#13361, @tico88612)
  • Disable controller-manager node CIDR allocation when using Cilium cluster-pool IPAM. (#13148, @guoard)
  • Ensure the netaddr prerequisite check always evaluates to a boolean. (#13304, @wt)
  • Fix k8s-certs-renew failing when super-admin.conf is missing. (#13414, @guoard)
  • Fix k8s-certs-renew renewing all control plane certificates on every timer run instead of only when they are close to expiry. (#13413, @guoard)
  • Fix Argo CD installation failures during password hashing. (#12920, @kwx4957)
  • Fix Calico IPv6-only deployments by disabling the IPv4 pool and IPv4 address assignment when IPv4 is disabled. (#13225, @AbhiiTen)
  • Fix Cilium startup when cilium_tunnel_mode: disabled by selecting native routing instead of passing an invalid tunnel protocol. (#13427, @mehrdadbn9)
  • Fix cluster setup in environments that require an HTTP proxy for external connectivity. (#13234, @modzilla99)
  • Fix false image-download failures when using nerdctl with download_run_once. (#12321, @AMacedoP)
  • Fix inverted error handling for etcd-events startup: healthy scale-outs no longer fail spuriously, and unhealthy clusters no longer have startup errors silently ignored. (#13343, @AdeshDeshmukh)
  • Fix kubeadm operations when the API server advertise address cannot be derived from the default gateway. (#13156, @clwluvw)
  • Fix non-boolean host-group conditions that prevent etcd and control-plane recovery from running with ansible-core 2.19. (#13541, @yankay)
  • Fix retries after a failed control-plane join by resetting the partial kubeadm state and handling leftover manifest and kubelet-port checks. Skip the post-join Ready wait when kube_network_plugin is cni or none, where CNI is installed separately. (#13280, @Ujstor)
  • Fix sample inventory resource reservations by moving group_vars/k8s_cluster/kube_control_plane.yml to group_vars/kube_control_plane.yml, so the higher reservations apply only to control-plane nodes rather than all cluster nodes. (#13518, @GmrezaH)
  • Fix skipped Calico IPv6 pool validation and prevent IPv6 pool discovery from overwriting the IPv4 pool fact on dual-stack clusters. (#13450, @mehrdadbn9)
  • Honor dns_autoscaler_deployment_nodeselector in the CoreDNS autoscaler deployment. (#13290, @Kibakus)
  • Offline containerd download URLs in the sample inventory now automatically use the correct binary release variant based on containerd_static_binary. (#13293, @EryX0)
  • Preserve the original YAML indentation when updating the server field in kubelet.conf, preventing invalid kubeconfigs after endpoint changes. (#13284, @longxiucai)
  • Reject incompatible combinations of Cilium <1.20, Gateway API >=1.5.0 standard-channel CRDs, and cilium_gateway_api_enabled: true before Cilium fails to start. For older Cilium versions, use gateway_api_version: "1.4.1" or gateway_api_channel: "experimental". (#13223, @yankay)
  • Remove redundant CNI role dependencies for Flannel, kube-router, macvlan, and Multus, fixing remove-node.yml failures caused by missing cached CNI plugin archives. (#12766, @11qeq11)
  • Remove the unnecessary BPF filesystem entry from /etc/fstab when using Cilium. (#13377, @wangsifei99)
  • Restore configurable fatal-error handling with kubespray_any_errors_fatal. Use -e '{"kubespray_any_errors_fatal": false}' to override the default; the old name, any_errors_fatal, conflicted with a reserved Ansible play keyword. (#13488, @gengwg)
  • Set numeric priority for static pods instead of priorityClassName so kubelet assigns the intended priority. (#13490, @jskrill)
  • Skip kube-proxy-only kernel modules, package selection, and IPVS/nftables checks when kube_proxy_remove: true. The default remains false. (#13228, @isumitsolanki)
  • Stop deleting the cert-manager namespace on upgrade runs, preserving user-managed resources such as Issuer secrets. (#13493, @longxiucai)
  • Support Kata Containers .zst release artifacts and update the default version. (#13389, @ChillRedChilli)
  • Update Local Path Provisioner to 0.0.37, including RBAC and helper-pod scheduling fixes. (#13525, @vashian)
  • Use containerd config version 4 with containerd 2.3 and newer. (#13286, @longxiucai)

Other (Cleanup or Flake)

Component Versions

  • Ansible 12.3.0 (ansible-core >=2.19.0,<2.20.0)
  • Kubernetes 1.36.4
  • etcd 3.6.14
  • Docker 28.3
  • containerd 2.3.5
  • CRI-O 1.36.5
  • runc 1.4.3
  • CNI plugins 1.9.1
  • Calico 3.31.7
  • Cilium 1.20.2
  • Gateway API 1.6.2
  • Flannel 0.28.9
  • Kube-OVN 1.12.21
  • Kube-router 2.1.1
  • Multus 4.2.2
  • kube-vip 1.0.3
  • cert-manager 1.15.3
  • CoreDNS 1.14.2
  • DNS autoscaler 1.10.3
  • Argo CD 2.14.5
  • Helm 3.22.0
  • MetalLB 0.13.9
  • Registry 2.8.1
  • nerdctl 2.3.5
  • metrics-server 0.9.0
  • Kata Containers 3.32.0
  • AWS EBS CSI plugin 0.5.0
  • Azure CSI plugin 1.10.0
  • Cinder CSI plugin 1.30.0
  • GCP PD CSI plugin 1.9.2
  • Local Path Provisioner 0.0.37
  • Local Volume Provisioner 2.5.0
  • Node Feature Discovery 0.19.0

New Contributors

Welcome and thank you to all the first-time contributors to Kubespray!

@11qeq11 @A-Soltani255 @AbhiiTen @AdeshDeshmukh @AlexandrClick @arhuman @ChillRedChilli @Debasish-87 @dhakshin-raghav @emma034186-stack @EryX0 @gengwg @isumitsolanki @jskrill @jsuchome @Kibakus @kwx4957 @longxiucai @manzsolutions-aleski @mehrdadbn9 @modzilla99 @Narwhal-fish @NikKumar811 @razvan01 @rtsui-harmonicinc @sandaruwijewardhana @shashankvarma499 @somaz94 @stias @suii2210 @taiko19xx @tophac @vashian @wt @xoxonut @Zakharden

All Contributors

Thank you to everyone who contributed to this release!

@11qeq11 @A-Soltani255 @AbhiiTen @AdeshDeshmukh @AlexandrClick @AMacedoP @arhuman @ChillRedChilli @clwluvw @Debasish-87 @dhakshin-raghav @emma034186-stack @ErikJiang @EryX0 @felipe88alves @gengwg @GmrezaH @guoard @isumitsolanki @Jeroen0494 @jmeza-xyz @jskrill @jsuchome @Kibakus @kwx4957 @longxiucai @manzsolutions-aleski @MatthiasLohr @mehrdadbn9 @modzilla99 @Narwhal-fish @NikKumar811 @razvan01 @rptaylor @rtsui-harmonicinc @sandaruwijewardhana @shaleenbathla @shashankvarma499 @somaz94 @Srishti-j18 @stias @suii2210 @taiko19xx @tico88612 @tmurakam @tophac @Ujstor @VannTen @vashian @wangsifei99 @wt @xoxonut @yangminglintw @yankay @Zakharden

Check out latest releases or
releases around kubernetes-sigs/kubespray v2.32.0

Don't miss a new kubespray release

NewReleases is sending notifications on new releases.

Get notifications