[gardener/gardener]

📰 Noteworthy

• [OPERATOR] Five minutes Infrastructure Cleanup Wait Period during shoot deletion was removed. Shoot annotation shoot.gardener.cloud/infrastructure-cleanup-wait-period-seconds which could be used to configure this period was removed, too. by @oliver-goetz [#9632]
• [DEVELOPER] The tools installed via the tools.mk make file are now by default installed in an OS and arch specific folder to allow running make targets from different platforms sharing the same source code.
  The previous behavior can be achieved by setting the variable TOOLS_BIN_DIR to hack/tools/bin to any make target. by @vpnachev [#9589]
• [DEVELOPER] Today's method of providing Plutono dashboards for garden or shoot clusters is deprecated and will be removed in a future release. Migrate to the new approach (see this document) for details. by @rfranzke [#9624]

✨ New Features

• [OPERATOR] gardener-operator is now managing the Gardener Dashboard web terminal controller manager when .spec.virtualCluster.gardener.gardenerDashboard.terminal is set in the Garden resource. Read more about it here by @rfranzke [#9646]
• [OPERATOR] gardener-node-agent no longer watches all Nodes in the cluster but restricts to only the Node it is responsible for (with the help of label/field selectors). This should lead to a significant reduction of network I/O, especially for shoot clusters with many nodes. by @rfranzke [#9672]
• [OPERATOR] gardener-operator now deploys two more Prometheus replicas into the garden namespace for storing long-term metrics. Read more about it here. by @rfranzke [#9606]
• [OPERATOR] A new feature gate named VPAForETCD is now introduced for gardenlet and gardener-operator. When enabled, VPA for etcd is used, regardless of the HVPA feature gate setting. The new VPA limits scaling down to a Shoot's maintenance window or even entirely based on the ShootClass in the same way as it is currently done for HVPA. by @voelzmo [#8984]
• [OPERATOR] gardener-operator is now managing the Gardener Dashboard when .spec.virtualCluster.gardener.gardenerDashboard is set in the Garden resource. Read more about it here by @rfranzke [#9583]
• [USER] It is now possible to define a higher number of maximum worker count in a shoot than pods and nodes networks allow. cluster-autoscaler ensures that not more nodes than the networking settings allow will be created. by @oliver-goetz [#9599]

🐛 Bug Fixes

• [OPERATOR] gardener-operator is now capable of reconciling shoot cluster-specific NetworkPolicys in case the garden cluster is a seed cluster at the same time. by @rfranzke [#9658]
• [OPERATOR] Fixed prometheus alerting rules for Seeds with unhealthy control-planes by @voelzmo [#9692]
• [OPERATOR] In the migrate flow of control plane migration the Deleting extensions before kube-apiserver task now depends on the Waiting until extension resources have been deleted task. by @plkokanov [#9651]
• [OPERATOR] Only update network policy allow-to-runtime-apiserver after resolver has been synced. by @MartinWeindel [#9644]

🏃 Others

• [OPERATOR] Updated VPA to 1.1.1 by @voelzmo [#8984]
• [OPERATOR] If a previous file copy attempt failed gardener-node-agent now deletes leftover *.tmp files instead of returning an error. by @oliver-goetz [#9630]
• [OPERATOR] extension library: An issue causing the backup.gardener.cloud/created-by annotation not being added on existing etcd-backup Secrets is now fixed. by @ialidzhikov [#9613]
• [OPERATOR] Added a cleanup function to gardenlet which is executed at startup and deletes orphaned VPAs with label role: vali-vpa that were previously managed by the HVPA deployed for vali. by @plkokanov [#9681]
• [OPERATOR] The gardenlet now runs as nonroot user and group 65532. by @AleksandarSavchev [#9669]
• [OPERATOR] A new plutono dashboard named Resource usage by container is added to garden/plutono. It shows aggregated CPU/memory usage vs requests/limits and utilization per container (currently only metrics for kube-apiserver containers are federated). by @ialidzhikov [#9643]
• [OPERATOR] Containers, configured to run as non-root, are now validated to start with non-root user by the kubelet. by @AleksandarSavchev [#9640]
• [OPERATOR] The fluent-operator component now runs as nonroot user and group 65532. by @AleksandarSavchev [#9640]
• [OPERATOR] The kube-controller-manager's (H)VPA minAllowed memory is reduced from 100Mi to 50Mi. The kube-apiserver's HVPA minAllowed memory is reduced from 400M to 200M. by @ialidzhikov [#9654]

Docker Images

• admission-controller: europe-docker.pkg.dev/gardener-project/releases/gardener/admission-controller:v1.94.0
• apiserver: europe-docker.pkg.dev/gardener-project/releases/gardener/apiserver:v1.94.0
• controller-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/controller-manager:v1.94.0
• gardenlet: europe-docker.pkg.dev/gardener-project/releases/gardener/gardenlet:v1.94.0
• node-agent: europe-docker.pkg.dev/gardener-project/releases/gardener/node-agent:v1.94.0
• operator: europe-docker.pkg.dev/gardener-project/releases/gardener/operator:v1.94.0
• resource-manager: europe-docker.pkg.dev/gardener-project/releases/gardener/resource-manager:v1.94.0
• scheduler: europe-docker.pkg.dev/gardener-project/releases/gardener/scheduler:v1.94.0