[gardener]

⚠️ Breaking Changes

• [USER] When the Gardener operators enable the ShootMaxTokenExpirationOverwrite feature gate then values for the .spec.kubernetes.kubeAPIServer.serviceAccountConfig.maxTokenExpiration field in the ShootSpec not in [30d,90d] will be overwritten to be within these boundaries. When they enable the ShootMaxTokenExpirationValidation feature gate then values in [30d,90d] are enforced. Adapt your shoot specifications to match these requirements! (gardener/gardener#5550, @rfranzke)
• [OPERATOR] The ShootExtensionStatus resource is no longer served from the core.gardener.cloud resource group by the gardener-apiserver. The resource was intended to hold information of the provider status fields from extensions resources from the Seed cluster but actually a controller acting on this resource was never added. (gardener/gardener#5618, @ialidzhikov)
• [OPERATOR] If you maintain ResourceQuota objects in the endusers' Project namespaces, make sure to increase the secrets quota, so that the new <shoot-name>.ca-cluster secret can be synced to the garden cluster (see documentation). (gardener/gardener#5612, @timebertt)
• [DEVELOPER] Remove all landscaper related code. (gardener/gardener#5481, @danielfoehrKn)
• [DEPENDENCY] Extensions using the token requestor (and hence the generic-token-kubeconfig secret) should switch to using extensionscontroller.GenericTokenKubeconfigSecretNameFromCluster in order to extract the name of the correct secret. This is a prerequisite for CA rotation. (gardener/gardener#5510, @rfranzke)

✨ New Features

• [USER] There is a new Secret for each Shoot in the corresponding Project Namespace (<shoot-name>.ca-cluster) which contains the current CA bundle for establishing trust to the Shoot's API server (see documentation). (gardener/gardener#5612, @timebertt)
• [OPERATOR] Logs of the gardener components in the shoot's kube-system are scraped and available for the operators. (gardener/gardener#5600, @vlvasilev)
• [OPERATOR] Allow the seed-prometheus to scrape pods labeled with networking.gardener.cloud/from-prometheus: allowed (gardener/gardener#5582, @voelzmo)
• [OPERATOR] Allow the seed-prometheus to scrape VPA recommender and VPA updater (gardener/gardener#5582, @voelzmo)
• [OPERATOR] The 'apiserver_audit_(event|error)_total' metrics of the shoot clusters are now preserved in the aggregated prometheus of the seed. (gardener/gardener#5573, @vpnachev)
• [OPERATOR] It is now possible to exclude ManagedResources from reconciliation by annotating the resources with resources.gardener.cloud/ignore=true. (gardener/gardener#5556, @rfranzke)
• [OPERATOR] A new controller in the gardenlet for syncing Secrets in shoot namespaces to ShootState resources has been introduced. It persists all marked secrets so that they can be used for restoration in case of a disaster or a control plane migration. (gardener/gardener#5503, @rfranzke)
• [OPERATOR] The storage capacity of the central Loki is now configurable (via the gardenlet's component config). The default storage capacity is increased from 30Gi to 100Gi. (gardener/gardener#5390, @vlvasilev)
• [DEVELOPER] A new manager for secrets related to seed or shoot clusters has been introduced. Please consult the documentation for more information. (gardener/gardener#5503, @rfranzke)

🐛 Bug Fixes

• [USER] A bug preventing the nodeTemplate in Machines to be updated when the machine type was changed has been fixed. (gardener/gardener#5577, @himanshu-kun)
• [USER] A race condition has been fixed which can lead to pods without any projected token volumes for newly created shoots. (gardener/gardener#5549, @rfranzke)
• [USER] A bug causing shoot reconciliations or deletions to fail with "no matches for kind" errors has been fixed. (gardener/gardener#5539, @rfranzke)
• [OPERATOR] The CheckDaemonSet func does no longer return err for a DaemonSet that is in ongoing rollout and has allowed number of unavailable replicas during the rollout. (gardener/gardener#5628, @ialidzhikov)
• [OPERATOR] An issue causing update request to SecretBinding with provider=nil to wrongly be rejected when the SecretBindingProviderValidation feature gate is enabled is now fixed. (gardener/gardener#5617, @ialidzhikov)
• [OPERATOR] An issue has been fixed leading to shoot namespaces in the seed blocking deletion due to referenced objects with finalizers. (gardener/gardener#5557, @rfranzke)
• [OPERATOR] An issue causing Shoot deletion to fail in a rare case when the corresponding Shoot Namespace in the Seed is already terminating is now fixed. (gardener/gardener#5555, @ialidzhikov)
• [OPERATOR] preserve service annotations for nginx-ingress-controller and istio-ingressgateway services (gardener/gardener#5457, @FlorinPeter)
• [DEVELOPER] Fixed an indentation issue in the VPA charts which caused a validation error when executing ./hack/check-charts.sh ./charts (gardener/gardener#5615, @voelzmo)
• [DEVELOPER] The helm version is now updated to v3.6.3 to prevent make install-requirements from failing on M1 Macs. (gardener/gardener#5546, @briantopping)
• [DEPENDENCY] A bug in the extensions health check library has been fixed which could prevent status reporting for the Worker resources. (gardener/gardener#5589, @rfranzke)

📖 Documentation

• [USER] Added documentation about enabling the CopyEtcdBackupsDuringControlPlaneMigration feature gate so that etcd backups are copied to the destination seed's BackupBucket during control plane migration. (gardener/gardener#5620, @plkokanov)
• [OPERATOR] The feature gate documentation does now contain information about which of the feature gates are relevant for which Gardener components. (gardener/gardener#5535, @rfranzke)
• [DEVELOPER] Added documentation about using the owner check mechanism introduced for the "bad case" scenario of control plane migration when implementing Reconcilers for new extension controllers. (gardener/gardener#5620, @plkokanov)

🏃 Others

• [OPERATOR] The file permissions of the keys in vpn-shoot are now properly set so that openvpn will not issue warnings. (gardener/gardener#5614, @ScheererJ)
• [OPERATOR] SeedKubeScheduler: gardenlet does now configure gardener-kube-scheduler running on K8s 1.23 Seed clusters with KubeSchedulerConfiguration from the kubescheduler.config.k8s.io/v1beta3 API version. (gardener/gardener#5584, @ialidzhikov)
• [OPERATOR] The Golang version was bumped to 1.17.8. (gardener/gardener#5575, @ialidzhikov)
• [OPERATOR] The kubectl get secretbinding table view was adapted to show the provider type field of the SecretBinding resource. (gardener/gardener#5566, @ialidzhikov)
• [OPERATOR] Increased the static memory limit of kube-proxy for cases where the vertical pod autoscaler is not acting as planned. (gardener/gardener#5552, @ScheererJ)
• [OPERATOR] SeedKubeScheduler: gardenlet does now configure gardener-kube-scheduler running on K8s 1.22 Seed clusters with KubeSchedulerConfiguration from the kubescheduler.config.k8s.io/v1beta2 API version. (gardener/gardener#5538, @ialidzhikov)
• [OPERATOR] The pods grafana dashboard now includes the node name and the pod/node ips per pod as well as a link to the node dashboard. (gardener/gardener#5537, @ScheererJ)
• [OPERATOR] The systemd services deployed to each shoot cluster worker node do no longer LIST nodes calls. Instead, the name of the node is fetched once and then stored in a file on the disk so that the systemd services can do GET node calls with the respective name of the node. This should reduce the load on the kube-apiserver and etcd. (gardener/gardener#5529, @rfranzke)

📰 Noteworthy

• [USER] There is a new section in the ShootStatus under .status.credentials.rotation.sshKeypair describing when the SSH keypair rotation was last initiated and last completed. (gardener/gardener#5583, @rfranzke)
• [USER] There is a new section in the ShootStatus under .status.credentials.rotation.kubeconfig describing when the kubeconfig rotation was last initiated and last completed. (gardener/gardener#5524, @rfranzke)
• [OPERATOR] There are two new feature gates affecting the values for the .spec.kubernetes.kubeAPIServer.serviceAccountConfig.maxTokenExpiration field in the ShootSpec: (gardener/gardener#5550, @rfranzke)
  • ShootMaxTokenExpirationOverwrite - if enabled then the gardener-apiserver overwrites any values for .spec.kubernetes.kubeAPIServer.serviceAccountConfig.maxTokenExpiration which are not in [30d,90d] to the respective boundary
  • ShootMaxTokenExpirationValidation - if enabled then the gardener-apiserver enforces that values for .spec.kubernetes.kubeAPIServer.serviceAccountConfig.maxTokenExpiration are in [30d,90d]
  • It is recommended to first enable ShootMaxTokenExpirationOverwrite to not break users specifying other values, and after some time enable ShootMaxTokenExpirationValidation to enforce the boundaries are respected. This is required to ensure all Gardener system components remain functional now that they leverage auto-rotated tokens requested by the TokenRequest API.
• [OPERATOR] The DNSRecord extension resources for shoot clusters are now only reconciled during shoot creation or maintenance or when they are unhealthy. Similarly, the DNSRecord extension resource for seed cluster is now only reconciled during seed creation or when it is unhealthy. Both is to prevent flooding DNS provider APIs which typically have quite low rate limits. (gardener/gardener#5531, @rfranzke)

[autoscaler]

🏃 Others

• [USER] CA now balances between similar node groups properly during scale-from-zero. (gardener/autoscaler#114, @himanshu-kun)
• [DEVELOPER] Gardener CA now supports basic IT to run locally. (gardener/autoscaler#111, @AxiomSamarth)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.43.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.43.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.43.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.43.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.43.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.43.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.43.0