[gardener]

⚠️ Breaking Changes

• [USER] New Shoots can no longer specify __internal for the apiVersion in their InfrastructureConfig. For compatibility reasons, existing Shoots with this configuration can still be updated. (gardener/gardener#4927, @voelzmo)
• [OPERATOR] The etcd backup compaction feature has been disabled by default. It can by enabled anytime via the etcdConfig section in the Gardenlet-Componentconfig. (gardener/gardener#5144, @timuthy)
• [OPERATOR] Before upgrading to this Gardener version make sure that your existing Gardener runs on at least v1.37. (gardener/gardener#5128, @rfranzke)
• [OPERATOR] gardener-apiserver's DisallowKubeconfigRotationForShootInDeletion feature gate that is GA since v1.36 is unconditionally enabled, and can no longer be specified in the gardener-apiserver's configuration. (gardener/gardener#5124, @acumino)
• [OPERATOR] The SeedAuthorizer and SeedRestriction features do no longer support "ambiguous" gardenlets (i.e., gardenlets responsible for multiple seed clusters) since this feature was dropped already with Gardener v1.27. In case you have activated these features then you have to make sure that you deploy a dedicated gardenlet per seed cluster and that they don't use a client certificate with the (now removed) gardener.cloud:system:seeds:<ambiguous> common name before updating to this Gardener version. This document describes how to make the gardenlet regenerate its client certificate after you have reconfigured it. (gardener/gardener#5093, @rfranzke)

✨ New Features

• [OPERATOR] The error code detection has been enhanced for ManagedResource objects that are stuck when a shoot is deleted. This enables Gardener to assign the corresponding error code(s) to the shoot object. (gardener/gardener#5111, @timuthy)
• [OPERATOR] Two fields LeaseDurationSeconds and LeaseResyncSeconds added under SeedControllerConfiguration to make Seed lease and duration configurable. Both field have default value of 2 seconds. (gardener/gardener#5092, @ary1992)
• [DEVELOPER] Support for the resources.gardener.cloud/preserve-resources annotation was added for Jobs, CronJobs, and DaemonSets. (gardener/gardener#5131, @rfranzke)
• [DEVELOPER] The TokenRequestor controller (part of gardener-resource-manager) can now optionally sync the tokens into a Secret in the target cluster (see this document for more information). (gardener/gardener#5084, @rfranzke)

🐛 Bug Fixes

• [OPERATOR] Gardener-Resource-Manager does not try overwrite the immutable field .spec.selector of Job objects anymore. (gardener/gardener#5167, @timuthy)
• [OPERATOR] An issue has been fixed that prevented etcd worker counts from being set correctly in the GardenletConfiguration. (gardener/gardener#5151, @timuthy)
• [OPERATOR] Fixed a bug that prevented shoot deletion with UseDNSRecords enabled if it was never previously reconciled with this feature gate enabled. (gardener/gardener#5135, @stoyanr)
• [OPERATOR] An issue causing the reconciliation of existing Shoot to be marked as Failed when the Secrets quota is exhausted is now fixed. (gardener/gardener#5126, @ialidzhikov)
• [OPERATOR] Endless waits are now avoided when deleting DNSProvider, DNSEntry, and DNSOwner resources. (gardener/gardener#5119, @stoyanr)
• [OPERATOR] A bug has been fixed which caused the gardener-controller-manager to hang forever in case the internal domain secret got deleted before the last ControllerRegistration. (gardener/gardener#5105, @rfranzke)
• [OPERATOR] Fix failing health check if loki is disabled in gardenlet configuration (gardener/gardener#5103, @dergeberl)
• [OPERATOR] A bug in the SeedAuthorizer has been fixed which allowed gardenlets to unconditionally delete BackupBuckets. (gardener/gardener#5091, @rfranzke)
• [DEVELOPER] The ManagedResource CRD in the /example dir was fixed. (gardener/gardener#5168, @timuthy)
• [DEVELOPER] The resources.gardener.cloud/preserve-resources annotation does now work properly for StatefulSets. (gardener/gardener#5131, @rfranzke)

📖 Documentation

• [USER] The usage document for NodeLocalDNS is now enhanced with explanation on effects during shoot reconciliation and recommendation for the same. (gardener/gardener#5070, @ashwani2k)
• [OPERATOR] Added figure to Gardener concept docs depicting how the more relevant resources like cloudprofile, seed, project, shoot`, etc. relate to each other. (gardener/gardener#5137, @vlerenc)
• [OPERATOR] A Gardener landscape operator can now consult to the SecretBinding ProviderController on how to enable the controller that helps on automatically setting the provider type of existing SecretBindings and on how to enable validation checks (required/immutable field) for the new provider type field. (gardener/gardener#5058, @ialidzhikov)

🏃 Others

• [USER] The SecretBinding resource does now contain a new field that denotes its provider type - provider.type. (gardener/gardener#5058, @ialidzhikov)
• [OPERATOR] Add rateLimit fields to CRD dnsproviders.dns.gardener.cloud (gardener/gardener#5165, @MartinWeindel)
• [OPERATOR] The namespace and pod labels are kept for the metrics in the seed-prometheus. (gardener/gardener#5148, @istvanballok)
• [OPERATOR] The gardener-resource-manager VPA does now specify minAllowed values to prevent too low resource recommendations from VPA that lead to OOM. (gardener/gardener#5116, @ialidzhikov)
• [OPERATOR] A final full snapshot is created and waited upon while copying backups for control plane migration "good case" and "bad case" scenarios. (gardener/gardener#5095, @stoyanr)
• [OPERATOR] Update istio to version 1.12.0. (gardener/gardener#5080, @DockToFuture)
• [OPERATOR] Added heatmaps for dns request latency to dns dashboards. (gardener/gardener#5078, @ScheererJ)
• [OPERATOR] The ManagedSeed controller does no longer add the Shoot cloud provider credentials into the Seed kubeconfig Secret (Seed .spec.secretRef). (gardener/gardener#5073, @ialidzhikov)
• [OPERATOR] Add access logs to istio ingress-gateway. (gardener/gardener#5071, @DockToFuture)
• [OPERATOR] gardener-controller-manager and gardenlet have started switching from logrus to zap. Make sure to use the json log format to have harmonized logging during the migration period. (gardener/gardener#5057, @timebertt)
• [DEVELOPER] The Golang version was bumped to 1.17.5. (gardener/gardener#5152, @ialidzhikov)
• [DEVELOPER] New check-docforge step will be executed on each PR in the CI/CD (gardener/gardener#5108, @Kristian-ZH)
• [DEVELOPER] The SeedAuthorizer does now allow DELETE requests if the resource does not exist in the system. (gardener/gardener#5091, @rfranzke)
• [DEVELOPER] When the gardenlet is locally started against the nodeless setup then a seed-specific client certificate is generated. This improves the accuracy of the SeedAuthorizer feature when it is activated. (gardener/gardener#5086, @rfranzke)

📰 Noteworthy

• [USER] In order to reduce the validity of client certificates used by kubelets running on the worker nodes of shoot clusters, the expiration duration for certificates issued via CertificateSigningRequests has been reduced from 1y to 30d. A custom expiration duration per CertificateSigningRequest can be set via the .spec.expirationSeconds fields (available from Kubernetes v1.22). (gardener/gardener#5096, @rfranzke)
• [OPERATOR] An official version skew policy document was added. You can take a look here. (gardener/gardener#5129, @rfranzke)
• [OPERATOR] All seed system components deployed by Gardener have been switched to projected ServiceAccount tokens (instead of continued usage of static tokens). (gardener/gardener#5128, @rfranzke)
• [OPERATOR] All shoot system components deployed by Gardener have been switched to projected ServiceAccount tokens (instead of continued usage of static tokens). (gardener/gardener#5099, @rfranzke)
• [OPERATOR] gardenlet and extension controllers do no longer use a client certificate but an auto-rotated ServiceAccount token which is only valid for 12h. (gardener/gardener#5012, @rfranzke)
• [OPERATOR] dependency-watchdog-probe does no longer use a client certificate but an auto-rotated ServiceAccount token which is only valid for 12h. (gardener/gardener#5011, @rfranzke)

[etcd-backup-restore]

🐛 Bug Fixes

• [OPERATOR] Took care of a bug where contexts were created multiple times without being cancelled. This caused a significant higher memory and CPU footprint. (gardener/etcd-backup-restore#409, @aaronfern)

[etcd-druid]

🏃 Others

• [OPERATOR] Updated labels used in compaction job to differentiate them from etcd pods. This allows for pod scheduling policies to schedule compaction jobs on predetermined nodes (gardener/etcd-druid#270, @aaronfern)

Docker Images

admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.38.0
apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.38.0
controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.38.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.38.0
landscaper-gardenlet: eu.gcr.io/gardener-project/gardener/landscaper-gardenlet:v1.38.0
scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.38.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.38.0
resource-manager: eu.gcr.io/gardener-project/gardener/resource-manager:v1.38.0