Deprecation / Removal
- Migrate node-role.kubernetes.io/master to node-role.kubernetes.io/control-plane (#10464, @unai-ttxu)
- Drop support for Kubernetes 1.25.x (move min version to 1.26.x) (#10420, @yankay)
- Drop installation notes for Debian Jessie (#10642, @jelmer)
Feature / Major Changes
- Make kubernetes v1.28.6 default (#10810, @mzaian)
- Add kubernetes v1.28.0, v1.28.1, v1.28.2, v1.28.3, v1.28.4, v1.28.5 hash (#10435, #10541, #10739, @mzaian ; #10390, @tmurakam ; #10624, @tmurakam)
- Add Retry for Applying PriorityClass (#10469, @hangscer8)
- Add option
crio_criu_support_enabled
to enable container forensic analysis (#10479, @tu1h) - Add option
kubectl_alias
to set bash alias of kubectl (#10552, @tu1h) - Add variable to configure ipvs modules (
kube_proxy_ipvs_modules
) (#10580, @borgiacis) - Check nameserver only when dns is enable (#10561, @yckaolalala)
- Correctly handle remove_default_searchdomains when value is undefined (#10533, @yckaolalala)
- Kube-scheduler: remove/update deprecated component component config v1beta3. (#10484, @mzaian)
- Terraform-aws: variable driven ami selection (
ami_name_pattern
/ami_virtualization_type
/ami_owners
) (#10520, @mertcancam) - Terraform-openstack: Added possibility to enable dhcp flag critical on one interface (#10446, @Xartos)
- This will introduce a new variable
kube_apiserver_admission_plugins_podnodeselector_default_node_selector
that can be used withkube_apiserver_admission_plugins_needs_configuration: [PodNodeSelector]
defined. So allows the users to configure PodNodeSelector plugin. (#10607, @titansmc) - UpCloud: Terraform provider updated to v2.12.0. Server groups with strict anti-affinity (move var from
anti_affinity_policy
toanti_affinity
) (#10474, @robinAwallace) - Update dockerfile to follow best practices (#10708, @maxime1907)
- Update to ansible 2.15 and set minimum version to 2.15.5 (#10481, @MrFreezeex)
- [etcd] Update Default etcd version to 3.5.10 for kubernetes 1.28, 1.27 and 1.26 (#10798, @VannTen)
- [etcd] update version to 3.5.9 for k8s 1.28 , 1.27 , 1.26 (#10482, @mzaian)
- [etcd] add 3.5.10 hashes (#10566, @mzaian)
- [vsphere_csi] Update to 3.1.0 supports Kubernetes Version 1.28 (#10451, @mzaian)
- [cinder_csi] Cinder-CSI now use
cluster_name
variable instead of the default hardcoded "kubernetes" value (#10422, @floryut)
Applications
- [argocd] update argocd to v2.8.4 (#10568, @mzaian)
- [helm] upgrade to 3.13.1 (#10567, @mzaian)
- [coredns] Added option coredns_additional_error_config to allow for configuration of the coredns error plugin. (#10501, @Elias-elastisys)
- [coredns] Support CoreDNS use host network & config CoreDNS port (#10617, @liuxu623)
- [coredns] Support disable dns autoscaler when use CoreDNS (#10608, @liuxu623)
- [coredns] Add pdb to coredns (#10557, @lobiyedKarim1)
- [cert-manager] upgrade to v1.13.2 (#10616, @liuxu623)
- [cert-manager] Upgrade to v1.12.6 (#10582, @chansuke)
- [cert-manager] Upgrade to v1.12.5 (#10500, @chansuke)
Network
- [cilium] Fix invalid hubble yaml if
cilium_hubble_tls_generate
is enabled (#10430, @toonalbers) - [cilium] Use correct ports in cilium metrics services if metrics are enabled. (#10519, @bakito)
- [cilium] Adds support for deploying clusters with cilium 1.14+ (#10684, @rl0nergan)
- [calico] Separate calico-node and calico-cni-plugin service accounts and update default calico to v3.26.1 (#10416, @mzaian)
- [calico] Use calico_pool_blocksize from cluster when existing (#10516, @VannTen)
- [calico] Update default calico to v3.26.3 (#10526, @mzaian)
- [calico] Update default calico to v3.26.4 (#10669, @mzaian)
- [kube-router] Default kube-router version updated to v2.0.0 (#10503, @bozzo)
- [kube-router] Default kube-router version updated to v1.6.0 (#10478, @bozzo)
- [kube-router] Add
kube_router_bgp_graceful_restart
optional setting for disabling graceful BGP restarts (default to true) (#10489, @rosskusler) - [metallb] Add option to set avoidBuggyIPs in IPAddressPools and change the default back to false (#10458, @zeeZ)
- [metallb] Metallb --lb-class cmd arg to support multiple LoadBalancer implementations (#10550, @Seal1998)
- [custom_cni] Add helm support for custom_cni deployment (#10529, @kukacz)
- [kube_vip] Add
kube_vip_lb_fwdmethod
option for kube-vip (#10762, @tu1h)
Container-Managers
- [containerd] Fix invalid version check in containerd jinja-template config (#10620, @khanhngobackend)
- [containerd] Make containerd 1.7.11 default (#10671, @mzaian)
- [containerd] Add hashes for containerd versions 1.7.6 ~ 1.7.8 default (#10439, #10525, #10589, @mzaian)
- [containerd] Specify the runc path when we use the containerd container engine and change the bin_dir path. (#10154, @qlijin)
- [containerd] Refactor NRI activation for containerd and CRI-O (remove
crio_enable_nri
andcontainerd_nri_disable
) now only one varnri_enabled
default to false (#10470, @fmuyassarov) - [containerd] Add Boolean option
enable_cdi
to enable cdi (false by default) (#10603, @krembu) - [containerd] Add configuration option for NRI (disable by default) in crio & containerd (using new
containerd_nri_disable
andcrio_enable_nri
) (#10454, @fmuyassarov) - [containerd] add config support
override_path
(#10776, @yankay) - [runc] Upgrade to v1.1.10 (#10671, @mzaian)
- [crio] Update to v1.28.1 (#10480, @qlijin)
- [crio] Remove crio package configuration during cleanup (#10584, @yckaolalala)
- [crio] Update docs for crio_registry_auth (#10785, @qlijin)
- [docker] Ability to define GPG key path for Docker APT (using new variable
docker_repo_key_keyring
) (#10513, @emiran-orange) - [kata-containers] Freshens configuration-qemu to latest template compatible with kata-containers 3.1.3. (#10466, @Alphadelta14)
- [nerdctl] Bump nerdctl version 1.7.1 (#10685, @yankay)
- [nerdctl] Change nerdctl version from 1.5.0 to 1.6.0 (#10475, @MaGaroo)
Documentation
- Add link to Cilium CNI documentation (#10431, @toonalbers)
- Update docs for calico_iptables_backend in Redhat/Centos.md (#10417, @yankay)
- Update metallb example configs (#10485, @caruccio)
- Updated AWS ALB ingress controller version (#10680, @kundan2707)
Bug or Regression
- Add a variable reset_restart_network_service_name in the reset role to be able to configure the name of the service which is restarted. (#10428, @RomainMou)
- Add dnsPolicy: ClusterFirstWithHostNet to DaemonSets with hostNetwork: true (#10618, @Payback159)
- Check for correct conntrack module presence, regardless of kernel versions (#10662, @VannTen)
- Fallback_ips: ignore unreachable hosts (#10601, @poblahblahblah)
- Fix 'kube-apiserver' tag inappropriately overwriting secrets at rest encryption token (#10460, @jwitko)
- Fix assertion for task item verify-settings (#10699, @piwinkler)
- Fix external-lb in kubelet.conf server address and kube-proxy api-server address (#10490, @ugur99)
- Fix forgotten update of etcd-servers list in apiserver manifest when scaling (#8253, @liupeng0518)
- Fix metallb example yaml (#10545, @caruccio)
- Fix reset job for cri-o container engine (#10197, @turbosnail)
- Fix restart network task cannot be skipped (ansible boolean conversion needed) (#10512, @ErikJiang)
- Fix: add kubelet tag in task of Fetch facts to avoid kubelet config inconsistencies (#10423, @NierYYDS)
- Fixes the path of the certificates use in the etcdctl.sh wrapper when the deployment type is not kubeadm. (#10467, @RomainMou)
- Hubble relay will work when cilium_cluster_name is customised. (#10614, @eugene-eeo)
- Disable podCIDR allocation from control-plane when using calico (#10639, @VannTen)
- Kubespray-defaults: Check for boostrap-os FQDN (#10590, @VannTen)
- Patch for modprobe_nf_conntrack for new Linux Kernel, when using ipvs (#10625, @abhishekkr)
- Remove always tag applied on bootstrap (#10556, @yckaolalala)
- Set remove_default_searchdomains to false by default (#10554, @hedayat)
- Swap is now disabled using systemd (mask of swap.target) (#10587, @VannTen)
- Fix undefined retries variable when copying etcdctl (#10634, @ErikJiang)
- Move control plane certs renewal "spread out" into the systemd timer (#10596, @VannTen)
- The dhcp configuration for dns nameservers are now the same than during installation (#10548, @smutel)
- Use correct env var name for kube-vip per service leader election (#10433, @ThisIsQasim)
- Don't fail on 304 Not Modified for an already downloaded file (#10452, @sathieu)
- Fix download retry when get_url has no status_code (#10613, @RomainMou)
- Fix ntp installation on SLES and openSUSE (#10786, @goldyfruit)
- Set the
maxUnavailable
of the coredns rolling update strategy to 1 (#10748, @tu1h) - Fix crio_version version comparison (#10780, @ledroide)
- Fix disable swap failed in Centos/RHEL 7 (#10751, @yankay)
- Fix image pull fail with insecure-registry (#10775, @yankay)
- Refactor check_galaxy + fix version (#10729, @VannTen)
- Fix Helm installation on SLES and openSUSE (#10794, @goldyfruit)
- Fix incorrect ciliumcli binary (#10575, @tu1h)
- Fix ntp installation on SLES and openSUSE (#10786, @goldyfruit)
- Fix the cluster installation on cluster using etcd clients nodes (cilium / calico / ...) (#10769, @VannTen)
Other (Cleanup or Flake)
- Cleanup a deprecation warning (ipaddr filter) (#10518, @VannTen)
- Decouple kubespray-defaults from download (#10626, @VannTen)
- Etcd/backup: use native ansible modules instead of shell (#10540, @VannTen)
- Etcd: use dynamic group for certs generation check (#10610, @VannTen)
- Factorize some identical playbooks steps into their own sub-playbooks (#10633, @VannTen)
- Pre-upgrade tasks cleanup (#10656, @VannTen)
- Refactor "multi" handlers to use listen (#10542, @VannTen)
- Remove unneeded workaround for removing kubeadm DNS (#10695, @VannTen)
- Removed DEPRECATED
--logtostderr
from metrics-server (#10709, @michaelkebe) - Update KUBESPRAY_VERSION for v2.23.1 (#10600, @yankay)
- Update several checksum for different modules & configuration (#10606, @mzaian)
- Use non-deprecated stdout_callback in CI (#10647, @VannTen)
- Validate systemd unit files when generating them (#10597, @VannTen)
- Using ctr pull instead of nerdctl to workaround #10670. (#10687, @yankay)
- Jinja syntax pre-commit validation (#10667, @VannTen)
- Bump vagrant version 2.3.7 (#10787, @yankay)
- Update KUBESPRAY_VERSION for v2.23.2 (#10800, @yankay)
Supported Components
- Core
- kubernetes v1.28.6
- etcd v3.5.10
- docker v20.10 (see note)
- containerd v1.7.11
- cri-o v1.27 (experimental: see CRI-O Note. Only on fedora, ubuntu and centos based OS)
- Network Plugin
- cni-plugins v1.2.0
- calico v3.26.4
- cilium v1.13.4
- flannel v0.22.0
- kube-ovn v1.11.5
- kube-router v2.0.0
- multus v3.8
- weave v2.8.1
- kube-vip v0.5.12
- Application
- cert-manager v1.13.2
- coredns v1.10.1
- ingress-nginx v1.9.4
- krew v0.4.4
- argocd v2.8.4
- helm v3.13.1
- metallb v0.13.9
- registry v2.8.1
- Storage Plugin
- cephfs-provisioner v2.1.0-k8s1.11
- rbd-provisioner v2.1.1-k8s1.11
- aws-ebs-csi-plugin v0.5.0
- azure-csi-plugin v1.10.0
- cinder-csi-plugin v1.22.0
- gcp-pd-csi-plugin v1.9.2
- local-path-provisioner v0.0.24
- local-volume-provisioner v2.5.0
Known issues
N/A