[gardener]
⚠️ Breaking Changes
- [OPERATOR] The
gardener-admission-controller
configuration API and http endpoints were changed in several aspects: (#3577, @timebertt)- the fields
server.https.tls.server{Cert,Key}Path
have been removed in favor ofserver.https.tls.serverCertDir
(the cert directory is expected to contain atls.crt
andtls.key
file) - metrics and health endpoints are now exposed as plain HTTP endpoints on dedicated ports (configurable via
server.{healthProbes,metrics}.port
- the
gardener-admission-controller
service included in Gardener's helm chart has a new named port (metrics
) for exposing the metrics endpoint - If you deploy this component/configuration manually, please adapt your usage accordingly. Gardener's helm charts were adapted to the changes.
- the fields
- [OPERATOR] The
.controllers.shootedSeedRegistration
field has been removed from theGardenletConfiguration
in favor of the newly introducedManagedSeed
controller (configurable via.controllers.managedSeed
). Please adapt your Gardenlet Helm chart values and/or example Gardenlet configuration files. (#3418, @stoyanr) - [DEVELOPER] Semantics of
controllerutils.{EnsureFinalizer,RemoveFinalizer}
were changed. Both funcs now usePATCH
requests instead ofUPDATE
andRemoveFinalizer
expects an additionalclient.Reader
for reading from the API server. (#3641, @timebertt)- Please use
controllerutils.{PatchFinalizers,PatchRemoveFinalizers}
preferably were applicable, if your controller is able to tolerate conflict errors tolerated by stale reads.
- Please use
- [DEVELOPER] The
.controllers.shootedSeedRegistration
field has been removed from theGardenletConfiguration
in favor of the newly introducedManagedSeed
controller (configurable via.controllers.managedSeed
). Please runmake dev-setup
or manually copyexample/20-componentconfig-gardenlet.yaml
over your old configuration file. (#3418, @stoyanr) - [DEPENDENCY] Semantics of
controllerutils.{EnsureFinalizer,RemoveFinalizer}
were changed. Both funcs now usePATCH
requests instead ofUPDATE
andRemoveFinalizer
expects an additionalclient.Reader
for reading from the API server. (#3641, @timebertt)extensioncontroller.{EnsureFinalizer,DeleteFinalizer}
have been removed in favor of the funcs incontrollerutils
.controllerutils.PatchFinalizers
was renamed toPatchAddFinalizers
.
- [DEPENDENCY] The mocks for Gardener packages were moved to dedicated folders in the respective package directories, i.e., if there is package
foo
in./pkg/path/to/foo
then the mock would be inpkg/path/to/foo/mock
instead of./pkg/mock/gardener/path/to/foo
. Only the mocks for third-party/vendored packages remain in./pkg/mock
. (#3640, @rfranzke) - [DEPENDENCY] The already deprecated packages
github.com/gardener/gardener/pkg/version
andgithub.com/gardener/gardener/pkg/version/verflag
are now removed. (#3626, @ialidzhikov)
✨ New Features
- [OPERATOR] It is now configurable for which shoot purposes the
BackupEntry
deletion grace period applies. An empty list (default) means that it applies for all shoot purposes (as it was earlier). If you want to only select specific purposes then please configure.controllers.backupEntry.deletionGracePeriodShootPurposes[]
in the gardenlet's component configuration. (#3637, @rfranzke) - [OPERATOR] CoreDNS deployment of shoot clusters can now be automatically restarted during the shoot's maintenance time window. This is used to solve problems with clients stuck to single replica of the deployment and thus overloading it. The feature can be enabled via the
ControllerManagerConfiguration
under.controllers.shootMaintecance.enableShootCoreAddonRestarter
(seeexample/20-componentconfig-gardener-controller-manager.yaml
). (#3596, @vpnachev) - [OPERATOR] An additional change detection mechanism for the file
download-cloud-config.sh
is now used to ensure the file is up-to-date even after VM reboot. (#3583, @vpnachev) - [OPERATOR] A new
Seed
reconciler was added to the Gardener-Controller-Manager. It creates a dedicated namespace per seed in the Garden clusterseed-<seed-name>
and copies common secrets from thegarden
Namespace (labelled withgardener.cloud/role
) to the seed namespace. Gardenlets are supposed to read secrets (or namespaced objects in general) from seed dedicated namespaces only in the future. (#3582, @timuthy) - [OPERATOR]
gardener-admission-controller
now exposes several metrics about its webhooks (e.g.controller_runtime_webhook_latency_seconds_bucket
,controller_runtime_webhook_requests_in_flight
andcontroller_runtime_webhook_requests_total
) (#3577, @timebertt)- The metric
gardener_admission_controller_invalid_webhook_requests_total
was removed in favor of the newly added metrics.
- The metric
- [OPERATOR]
Seed
resources now have a new condition typeBackupBucketsReady
that is added when the corresponding seed has a backup configuration or relatedBackupBuckets
.Seeds
whoseBackupBucketsReady
condition isstatus: "False"
are consideredNotReady
and thus are excluded from scheduling during that time. (#3531, @timuthy) - [OPERATOR] A new
ManagedSeed
resource and its corresponding controller have been added and the existing shooted seed registration controller has been reworked to use them. (#3418, @stoyanr)
🐛 Bug Fixes
- [USER] A potential
nil
pointer exception in theShoot
validation (leading to503
responses fromgardener-apiserver
) when validating PID reservations (e.g., inkubeReserved
orsystemReserved
) has been fixed. (#3632, @rfranzke) - [OPERATOR] Fixed nil pointer exception that occurs when there are still extension resources in the
Seed
, but theCluster
resource has been deleted. (#3622, @plkokanov) - [OPERATOR] Fix a bug where
cloud-config-downloder
systemd service is set toFailed
with statusstart-limit-hit
if it is requested to be restarted via the node annotationworker.gardener.cloud/restart-systemd-services
. (#3593, @vpnachev) - [OPERATOR] Fixed an issue with enabling
KonnectivtyTunnel
via annotation (alpha.featuregates.shoot.gardener.cloud/konnectivity-tunnel: "false"
) onAPIServerSNI
-enabled Seed cluster causing the tunnel to not be opened. (#3586, @mvladev) - [OPERATOR] An issue causing gardener-controller-manager to not be able to delete a Plant when the Plant Secret is not found is now fixed. (#3584, @ialidzhikov)
- [OPERATOR]
gardener-controller-manager
now waits for a project's namespace to be empty before continuing with releasing the namespace and deleting the project. (#3578, @timebertt)
🏃 Others
- [USER] The external DNS record for the kubernetes API server is now deleted after the kubernetes API server. This is useful for shoot cluster owners that need to clean some kubernetes resources that can cause the shoot cluster deletion to stuck. (#3576, @vpnachev)
- [OPERATOR] A new error code for retryable configuration problems (for example misconfigured PodDisruptoinBudget that does not allow voluntary Pod evictions) is now added. (#3665, @danielfoehrKn)
- [OPERATOR]
istiod
is now scaled automatically byVerticalPodAutoscaler
instead ofHorizontalPodAutoscaler
. This fixes OOMKilled issues on big Seed clusters. (#3613, @mvladev) - [OPERATOR] Gardener now deploys the Cluster-Autoscaler earlier during the shoot creation which enables self healing for creation failures due to over-provisioned small machines. (#3612, @timuthy)
- [OPERATOR] Node exporter provides the metric node_uname_info (#3587, @dschmo)
- [OPERATOR]
gardener-admission-controller
's webhooks now also accept reviews in versionadmission/v1
. Also, webhook timeouts have been lowered to10s
. (#3577, @timebertt) - [OPERATOR] Use PATCH to update the extensions' state and relevant resource data to the ShootState. (#3550, @plkokanov)
📰 Noteworthy
- [USER] When a shoot is erroring with
ERR_INFRA_INSUFFICIENT_PRIVILEGES
,ERR_INFRA_QUOTA_EXCEEDED
orERR_INFRA_DEPENDENCIES
then it is now immediately set to theFailed
status (this already happens also forERR_INFRA_UNAUTHORIZED
orERR_CONFIGURATION_PROBLEM
). This prevents Gardener from automatically retrying the operation. If you are hit by it, please manually retry the operation once you have resolved the issue. (#3669, @rfranzke) - [USER] When a shoot is erroring with
ERR_INFRA_INSUFFICIENT_PRIVILEGES
,ERR_INFRA_QUOTA_EXCEEDED
orERR_INFRA_DEPENDENCIES
then it is now immediately set to theFailed
status (this already happens also forERR_INFRA_UNAUTHORIZED
orERR_CONFIGURATION_PROBLEM
). This prevents Gardener from automatically retrying the operation. If you are hit by it, please manually retry the operation once you have resolved the issue. (#3662, @rfranzke) - [OPERATOR] Starting with Gardener v1.18, the
shoot.gardener.cloud/use-as-seed
annotation is deprecated. The newManagedSeed
resource should be used instead to register shoots as seeds. (#3579, @stoyanr) - [DEPENDENCY] In the generic worker actuator's migration flow, the MCM finalizer of the secret that is referenced by the
Worker
is not removed. We have now added functionality that checks that secret and removes only the MCM finalizers if necessary. (#3560, @kris94) - [DEPENDENCY] The
Terraformer
interface has now a new functionRemoveTerraformerFinalizerFromConfig
which will remove the "terraformer" finalizer from theSecret
/ConfigMap
resources. (#3556, @kris94)
[gardener-resource-manager]
⚠️ Breaking Changes
- [OPERATOR] Gardener-Resource-Manager now needs permission to
get
,create
,update
andwatch
Lease
objects namedgardener-resource-manager
in order to perform leader election. For a reference, please have a look at the pre-delivered Helm chart incharts/gardener-resource-manager
. (gardener/gardener-resource-manager#105, @timuthy)
✨ New Features
- [OPERATOR] Gardener-Resource-Manager now adds latest warning events to a ManagedResource's
.status.conditions
in case a KubernetesService
cannot be deleted. This allows to get more context about the underlying problem e.g., when Cloud-Controller-Manager cannot delete the backing load balancer. (gardener/gardener-resource-manager#106, @timuthy) - [OPERATOR] If a ManagedResource refers to a
Service
object of typeLoadBalancer
, the Gardener Resource Manager now regularly checks if theService
has anIngress
status and contributes the result of this check to theResourcesHealthy
condition. (gardener/gardener-resource-manager#106, @timuthy) - [OPERATOR] With this release for all objects managed by a
ManagedResource
an annotationresources.gardener.cloud/origin
is set describing theManagedResource
that caused this object to be created. The format of the origin annotation is[cluster id:]namespace/object-name
. For multi-cluster scenarios the GRM can be started with a--cluster-id
options to enable the extended annotation format (see https://github.com/gardener/gardener-resource-manager/blob/master/docs/concepts/managed-resource.md for further details). (gardener/gardener-resource-manager#89, @mandelsoft)
🐛 Bug Fixes
- [OPERATOR] A problem with long running ManagedResource reconciliations caused by unavailable
APIServices
was fixed. (gardener/gardener-resource-manager#112, @timebertt) - [OPERATOR] The client QPS and burst settings do now also apply for the uncached client. (gardener/gardener-resource-manager#111, @rfranzke)
- [OPERATOR] The
.spec.loadBalancerIP
value forService
s is now preserved. (gardener/gardener-resource-manager#108, @deitch)
🏃 Others
- [OPERATOR] The
CheckDaemonSet
function does now lead to more accurate results. (gardener/gardener-resource-manager#103, @rfranzke)
📰 Noteworthy
- [OPERATOR] The controller reconciliations are now limited to
1m
. (gardener/gardener-resource-manager#102, @rfranzke) - [DEVELOPER] Go dependencies have been updated to: (gardener/gardener-resource-manager#105, @timuthy)
- github.com/gardener/gardener v1.16.0
- sigs.k8s.io/controller-runtime v0.7.1
- k8s.io/* v0.19.6
Docker Images
gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.18.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.18.0
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.18.0
gardener-admission-controller: eu.gcr.io/gardener-project/gardener/admission-controller:v1.18.0
gardener-seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.18.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.18.0