[gardener]
⚠️ Breaking Changes
- [USER] Since
Namespace
s are no longer deleted (and forcefully finalized after some grace period), theshoot.gardener.cloud/cleanup-namespaces-finalize-grace-period-seconds
annotation does no longer have any effect. Relevant Kubernetes resources are still cleaned up (see this document) for more information. (gardener/gardener#7864, @rfranzke) - [USER] Using internal API versions in
providerConfig
fields is no longer permitted (deprecated since more than2y
). Ensure that you always use a versioned API. (gardener/gardener#7868, @rfranzke) - [USER] As of Kubernetes
v1.27
, Gardener enforces aworker.maximum
configuration for system component worker pools. The value must be greater or equal to the number of zones configured for this pool. This ensures, that the pool has the minimum required nodes to schedule system component across nodes. (gardener/gardener#7878, @timuthy) - [USER] The static token kubeconfig can no longer be enabled for Shoot clusters using Kubernetes version
1.27
and higher. (gardener/gardener#7883, @ary1992) - [USER] For Shoot clusters using Kubernetes version
1.27
and higher, the.spec.kubernetes.kubeControllerManager.podEvictionTimeout
field has no effect anymore since the backing--pod-eviction-timeout
CLI flag has been removed. (gardener/gardener#7883, @ary1992) - [USER] ⚠️ The deprecated field
.spec.kubernetes.kubeAPIServer.enableBasicAuthentication
has been removed from the Shoot API. Please check yourShoot
s manifests and remove the.spec.kubernetes.kubeAPIServer.enableBasicAuthentication
field. (gardener/gardener#7886, @dimitar-kostadinov) - [USER] Gardener denies setting
Shoot.Spec.ControlPlane.HighAvailability.FailureTolerance.Type
if shoot is hibernated. (gardener/gardener#7894, @aaronfern) - [OPERATOR] All
fluent-bit
-related configuration options have been removed fromgardenlet
's component configuration. (gardener/gardener#7568, @Kristian-ZH) - [OPERATOR] The
FullNetworkPoliciesInRuntimeCluster
feature gate has been promoted to beta and is now turned on by default. Before deploying this Gardener version, make sure that all your registered extensions support this feature gate. (gardener/gardener#7866, @rfranzke) - [OPERATOR] The
HAControlPlanes
feature gate has been promoted to beta and is now turned on by default. (gardener/gardener#7867, @timuthy) - [OPERATOR] The deprecated
allow-{to,from}-shoot-apiserver
NetworkPolicy
s have been dropped. Ensure that all registered extensions have been adapted. (gardener/gardener#7868, @rfranzke) - [OPERATOR] The deprecated
identity
value is no longer passed whenControllerInstallation
Helm charts are deployed. (gardener/gardener#7868, @rfranzke) - [OPERATOR] The
lastUpdateTime
of extension conditions is no longer considered. Ensure that all registered extensions populate thelastHeartbeatTime
field instead. (gardener/gardener#7868, @rfranzke) - [DEVELOPER] The
pkg/operation/botanist/component/*
resources have been moved topkg/component/*
. (gardener/gardener#7938, @rfranzke) - [DEVELOPER]
gardenlet
will no longer respectConfigMap
s labeled withextensions.gardener.cloud/configuration=logging
. The way to deploy a new filter or parser configuration is to createClusterFilter
s orClusterParser
s custom resources in the seed cluster. (gardener/gardener#7568, @Kristian-ZH) - [DEVELOPER] Extensions vendoring this
gardener/gardener
version need to provide RBAC privileges forPATCH apps/depoyments/scale
. (gardener/gardener#7868, @rfranzke) - [DEPENDENCY] Extensions that wish to be scraped by the
seed-prometheus
must annotate their pods withprometheus.io/scrape=true
along withprometheus.io/name=<name>
. See https://github.com/gardener/gardener/blob/master/docs/monitoring/README.md#seed-prometheus for more details. (gardener/gardener#7885, @shafeeqes)
✨ New Features
- [USER] It is possible now to create a workerless shoot cluster when the
WorkerlessShoots
feature gate in thegardener-apiserver
is enabled. Please see this document for more details. (gardener/gardener#7882, @shafeeqes) - [OPERATOR]
fluent-operator
is now installed in thegarden
namespace of seed clusters and will take care of the entire lifecycle of thefluent-bit
DaemonSet
. (gardener/gardener#7568, @Kristian-ZH) - [OPERATOR] The
gardener-operator
now enables fullNetworkPolicy
protection for the garden cluster. In case your garden cluster is a seed at the same time, make sure to keep the values of theFullNetworkPoliciesInRuntimeCluster
feature gate in sync for bothgardener-operator
andgardenlet
. (gardener/gardener#7859, @rfranzke) - [OPERATOR]
gardenlet
andgardener-operator
manageddeployment
s andstatefulset
s can now be equipped with toleration seconds for taintsnode.kubernetes.io/not-ready
andnode.kubernetes.io/unreachable
. (gardener/gardener#7861, @timuthy)- Please consult the respective component config examples (
gardenlet
,gardener-operator
) for more information.
- Please consult the respective component config examples (
- [OPERATOR] The
gardenlet
andgardener-operator
Helm charts allow to define toleration seconds fornode.kubernetes.io/not-ready
andnode.kubernetes.io/unreachable
. This configuration considered for their own Deployment as well as the Gardenlet's or Operator's config. The values are set to60s
by default. (gardener/gardener#7861, @timuthy) - [OPERATOR] An optional field
workerlessSupported
is added underspec.resources
in theControllerRegistration
API. (gardener/gardener#7863, @ary1992) - [OPERATOR]
gardener-operator
is now managing thegardener-resource-manager
instance as part of the virtual garden cluster control plane. It provides aTokenRequest
API-based kubeconfig forgardener-operator
to access the virtual garden cluster. The static token kubeconfig is now unconditionally disabled. (gardener/gardener#7881, @oliver-goetz) - [OPERATOR] It is now possible to provide namespace selectors for additional namespaces which should be covered by the
NetworkPolicy
controllers ofgardener-operator
orgardenlet
. The selectors must be provided via their component configs. Please consult this document for further insights. (gardener/gardener#7929, @rfranzke) - [OPERATOR]
gardener-operator
is now managing thekube-controller-manager
instance as part of the virtual garden cluster control plane. (gardener/gardener#7931, @rfranzke) - [DEVELOPER] In order to allow
kube-apiserver
pods of shoot or garden clusters to reach webhook servers, they must no longer be explicitly labeled withnetworking.resources.gardener.cloud/to-<service-name>-<protocol>-<port>=allowed
. Instead, it is enough to annotate theService
of the webhook server withnetworking.resources.gardener.cloud/from-all-webhook-targets-allowed-ports=<ports>
. (gardener/gardener#7907, @rfranzke) - [DEPENDENCY] To support workerless Shoots, extensions reconciling
extensions.gardener.cloud/v1alpha1.Extension
resources need to make adaptions if needed and then setspec.resources[].workerlessSupported
totrue
in theControllerRegistration
for their respective extension type. (gardener/gardener#7863, @ary1992)
🐛 Bug Fixes
- [USER] An issue has been fixed which might have caused the deletion of
Shoot
clusters to stuck when a namespace was forcefully removed before all relevant resources have been cleaned up. (gardener/gardener#7864, @rfranzke) - [USER] A bug has been fixed which could cause
kube-proxy
s from being missing after aShoot
has been woken up from hibernation. (gardener/gardener#7912, @rfranzke) - [OPERATOR] An issue causing
VPN Seed (CPU| Memory) Usage
dashboards not showing data is now fixed. (gardener/gardener#7865, @Sallyan) - [OPERATOR] A bug has been fixed which prevented components using the
networking.resources.gardener.cloud/from-world-to-ports
annotation from being reached from internal IP addresses when the cluster was using Cilium as CNI. (gardener/gardener#7884, @ScheererJ) - [OPERATOR] A bug which was causing race conditions to occur during reconciliation of extension resources was fixed. (gardener/gardener#7906, @dimityrmirchev)
- [OPERATOR] An issue causing panic in the health check for extension, when the health check result is empty, is fixed. (gardener/gardener#7908, @acumino)
- [OPERATOR] An issue has been fixed that caused traffic from outside of the cluster to
Istio-Ingress
being blocked. This is only relevant if seed(s) specify additional load balancer annotations viaseed.spec.settings.loadBalancerServices.annotations
. (gardener/gardener#7910, @timuthy)
📖 Documentation
- [DEVELOPER] A guideline for developers regarding
TODO
statements has been introduced. (gardener/gardener#7939, @rfranzke)
🏃 Others
- [USER] The
--node-monitor-grace-period
flag ofkube-controller-manager
is now defaulted to40s
for Shoot clusters using Kubernetes version1.27
and higher. (gardener/gardener#7883, @ary1992) - [USER] The following images are updated: (gardener/gardener#7897, @himanshu-kun)
eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler
:v1.21.5
->v1.21.6
(for Kubernetes1.21
)eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler
:v1.22.5
->v1.22.6
(for Kubernetes1.22
)eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler
:v1.23.3
->v1.23.4
(for Kubernetes1.23
)eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler
:v1.24.2
->v1.24.3
(for Kubernetes1.24
)eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler
:v1.25.2
->v1.25.3
(for Kubernetes1.24
)eu.gcr.io/gardener-project/gardener/autoscaler/cluster-autoscaler
:v1.26.1
->v1.26.2
(for Kubernetes1.26
)
- [OPERATOR] Default log level in fluent-bit is changed from
info
toerror
(gardener/gardener#7942, @nickytd) - [OPERATOR] Grafana and Loki are replaced with the fork of their last Apache 2.0 licensed releases: Plutono and Vali, that will continue to receive security updates. (gardener/gardener#7318, @istvanballok)
- [OPERATOR] The following image is updated: (gardener/gardener#7892, @rickardsjp)
- quay.io/prometheus/prometheus: v2.41.0 -> v2.43.1
- [OPERATOR]
nginx-ingress-controller-seed
image is updated tov1.7.1
for1.24.x+
seeds. (gardener/gardener#7904, @shafeeqes) - [OPERATOR] Allow the kubelet configuration to define swap behaviour {LimitedSwap / UnlimitedSwap} for k8s >= 1.22 (gardener/gardener#7913, @danielfoehrKn)
- [OPERATOR] Updated cluster-proportional-autoscaler to v1.8.8 (gardener/gardener#7927, @ScheererJ)
- [OPERATOR] The gardenlet and the gardener-operator will now use the new
service.kubernetes.io/topology-mode=auto
annotation when enabling topology-aware routing for a Service when the Kubernetes version of the runtime cluster is >= 1.27. In Kubernetes 1.27, theservice.kubernetes.io/topology-aware-hints=auto
annotation is deprecated in favor of the newly introducedservice.kubernetes.io/topology-mode=auto
(gardener/gardener#7933, @ialidzhikov) - [DEVELOPER] The
check-apidiff
check was changed to only report incompatible and critical changes which need inspection from the developer's side. (gardener/gardener#7936, @timuthy) - [DEVELOPER] The
networking.resources.gardener.cloud/from-policy-pod-label-selector
andnetworking.resources.gardener.cloud/from-policy-allowed-ports
annotations are now deprecated and will be removed in the future. Usenetworking.resources.gardener.cloud/from-<pod-label-selector>-allowed-ports=<ports>
instead. (gardener/gardener#7907, @rfranzke) - [DEPENDENCY] Shoot addon
nginx-ingress-controller
image is updated tov1.3.0
forv1.22+
shoots. (gardener/gardener#7932, @shafeeqes)
[apiserver-proxy]
🏃 Others
- [OPERATOR] Use admission v1 instead of v1beta1 for apiserver-proxy webhook. (gardener/apiserver-proxy#35, @ScheererJ)
- [OPERATOR] Fix verification. (gardener/apiserver-proxy#38, @axel7born)
📰 Noteworthy
- [OPERATOR] Update golang 1.19.5 -> 1.20.4 (gardener/apiserver-proxy#37, @axel7born)
[ext-authz-server]
✨ New Features
- [OPERATOR] Update golang 1.19.5 -> 1.20.4 (gardener/ext-authz-server#19, @axel7born)
[logging]
🏃 Others
- [OPERATOR] Prevent fluent-bit-to-vali plugin panic when Cluster is updated and its Shoot has no lastOperation set (gardener/logging#192, @vlvasilev)
- [OPERATOR] Improves client recreate during cluster reconcile. (gardener/logging#195, @nickytd)
- [OPERATOR] Update
k8s.io/client-go
from v0.17.0 to v0.26.2 (gardener/logging#188, @vlvasilev) - [OPERATOR] Updated golang container image build version to 1.20.4 (gardener/logging#190, @nickytd)
- [DEVELOPER] This PR aligns container build targets with project CI supporting multi-platform builds and simplifies overall Makefile structure. (gardener/logging#189, @nickytd)
[vpn2]
🏃 Others
- [OPERATOR] Bump builder image golang from
1.20.2
to1.20.4
(gardener/vpn2#32, @MartinWeindel) - [OPERATOR] Bump builder image golang from
1.19.5
to1.20.2
(gardener/vpn2#30, @MartinWeindel) - [OPERATOR] Bump alpine base image from
1.16.3
to1.16.5
(gardener/vpn2#30, @MartinWeindel) - [OPERATOR] Updated kubernetes dependencies from
1.25.0
to1.26.2
(gardener/vpn2#30, @MartinWeindel)
Docker Images
admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.71.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.71.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.71.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.71.0
operator: eu.gcr.io/gardener-project/gardener/operator:v1.71.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.71.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.71.0