[gardener]

Action Required

• [OPERATOR] The minimum required Kubernetes version for the garden cluster is now 1.12. Please make sure this your garden cluster is of at least this version before upgrading Gardener. (#2151, @swilen-iwanow)
• [DEVELOPER] Extension controllers using extensions/pkg/controller/error.RequeueAfterError need to make sure that the cause error is properly logged on their side. (#2351, @ialidzhikov)
• [DEVELOPER] The old testframework has been completely removed. All extension repositories that previously used the networkolicies generator have to be regenerated when this change is vendored. (#2344, @schrodit)
• [DEVELOPER] Extension controllers for Network and ContainerRuntime CRDs have to implement the Restore and Migrate operations. (#2276, @swilen-iwanow)

Most notable changes

• [OPERATOR] The Gardenlet does now run a HTTPS server that serves a /healthz and /metrics endpoint. You should generate a server certificate for the gardenlet, gardenlet.garden, gardenlet.garden.svc hosts. The bind address, port, and TLS certificate paths are configurable in its component config. Also, the gardenlet Helm chart was enhanced with a liveness probe that targets the /healthz endpoint. (#2309, @rfranzke)
• [USER] The SecurityContextDeny admission plugin is no longer allowed to be used for shoots as it conflicts with the PodSecurityPolicy admission plugin which is enabled by default. (#2346, @rfranzke)
• [USER] Shoot clusters which are flagged with ERR_INFRA_UNAUTHORIZED or ERR_CONFIGURATION_PROBLEM error codes are now automatically set to Failed status. This means that they won't be retried automatically unless you annotate the Shoot with gardener.cloud/operation=retry. All other error codes will lead to automatic retries for at most 12h before the shoot is set to Failed. (#2333, @rfranzke)
• [USER] Hibernation blocking due to Mutating/Validating webhooks is now improved and enforced for hooks with failurePolicy: Fail and operation CREATE, UPDATE or * for the following resources: (#2270, @mvladev)
  • apiservices
  • apiservices/status
  • certificatesigningrequests
  • certificatesigningrequests/approval
  • certificatesigningrequests/status
  • clusterrolebindings
  • clusterroles
  • configmaps (only for kube-system namespace)
  • controllerrevisions (only for kube-system namespace)
  • customresourcedefinitions
  • customresourcedefinitions/status
  • daemonsets (only for kube-system namespace)
  • daemonsets/status (only for kube-system namespace)
  • deployments (only for kube-system namespace)
  • deployments/scale (only for kube-system namespace)
  • endpoints
  • leases
  • namespaces
  • namespaces/finalize
  • namespaces/status
  • networkpolicies (only for kube-system namespace)
  • nodes
  • nodes/status
  • pods (only for kube-system namespace and shoot.gardener.cloud/no-cleanup=true,orgin=gardener labels)
  • pods/status (only for kube-system namespace and shoot.gardener.cloud/no-cleanup=true,orgin=gardener labels)
  • podsecuritypolicies
  • priorityclasses
  • replicasets (only for kube-system namespace)
  • replicasets/scale (only for kube-system namespace)
  • replicasets/status (only for kube-system namespace)
  • rolebindings (only for kube-system namespace)
  • roles (only for kube-system namespace)
  • secrets (only for kube-system namespace)
  • serviceaccounts (only for kube-system namespace)
  • services
  • services/status
• [OPERATOR] The default value for retryDuration has been changed from 24h to 12h. Hence, Gardenlet tries to reconcile shoots with erroneous operations for 12 hours (by default). After this period of time only the retry operation, a .spec change, or a rollout of a new Gardenlet version re-triggers a reconciliation. (#2324, @timuthy)
• [OPERATOR] The following taints have been deprecated and will be removed in a future version: (#2315, @rfranzke)
  • seed.gardener.cloud/disable-capacity-reservation in favour of the new .spec.settings.excessCapacityReservation.enabled field.
  • seed.gardener.cloud/invisible in favour of the new .spec.settings.scheduling.visible field.
  • seed.gardener.cloud/disable-dns in favour of the new .spec.settings.shootDNS.enabled field.
  • The .controllers.seed.reserveExcessCapacity setting in the component config of the Gardenlet has been removed in favour of the new settings field mentioned earlier.
  • Please check this document for further information about the seed settings.
• [OPERATOR] The ControllerRegistration object does now allow better control for deployment/deletion of extension controllers to seed clusters (policy, seed selector, ...). You might want to look into this document. (#2278, @rfranzke)
• [OPERATOR] Added new feature gate ManagedIstio to gardenlet. When enabled it deploys a customized installation of istio on Seed clusters. Disabling it once enabled does not remove any installed resources. (#2273, @mvladev)
  • The installation deploys istiod in the istio-system namespace and istio ingress gateway in istio-ingress namespace. mTLS is enforced and Services, VirtualServices and DestinationRules are not exported and advertised by default and therefore it must be explicitly enabled either via the networking.istio.io/exportTo: "*" annotation or .exportTo: ["*"].

Improvements

• [USER] An issue causing an additional reconciliation after successful flow execution for newly created Shoots is now fixed. (#2364, @ialidzhikov)
• [USER] A bug has been fixed that caused the gardener-controller-manager to incorrectly delete extension ClusterRoles of other projects when reconciling a Project that doesn't have a member with the same extension role. (#2352, @rfranzke)
• [USER] The Shoot specification does now have a new .spec.seedSelector field which allows to provide label selector. Only seeds whose labels match will be considered for scheduling decisions. (#2340, @rfranzke)
• [USER] The bootstrapping of new shoot worker nodes has been made more reliable. (#2313, @rfranzke)
• [USER] It is now possible to opt-out of/disable globally enabled extensions for Shoots by setting .spec.extensions[] = {type: <extension-type>, disabled: true}. (#2278, @rfranzke)
• [OPERATOR] Some grafana dashboards have been changed to use a rate of 2m instead of 1m. (#2386, @wyb1)
• [OPERATOR] A bug has been fixed that made the gardener-scheduler considering protected seeds for shoots outside the garden namespace, potentially making it trying to schedule a shoot to such a seed (which will fail forever). (#2382, @rfranzke)
• [OPERATOR] Now when the excess capacity reservation is disabled, the reservation pods are deleted. (#2365, @vpnachev)
• [OPERATOR] Fixed a bug when deploying ContainerRuntime custom resources for multiple worker pools. (#2357, @danielfoehrKn)
• [OPERATOR] Extension Reconcilers do no longer log the same error which the controller-runtime is supposed to log too. This should prevent given error to appear multiple times in the logs. (#2351, @ialidzhikov)
• [OPERATOR] Resources (usually secrets) required by extension controllers can now be added to the extension status, persisted in the ShootState, and migrated from the source seed to the destination seed. (#2350, @stoyanr)
• [OPERATOR] A bug that was preventing the Gardenlet from bootstrapping seed clusters if the HVPA feature gate is disabled was fixed. (#2345, @rfranzke)
• [OPERATOR] It is now possible to add arbitrary labels to the .spec.regions[].labels field in the CloudProfile, e.g. to provide more information about reliability, access restrictions, etc. (#2340, @rfranzke)
• [OPERATOR] The check for stale extension healthchecks is now also disabled by default in gardenlet chart values.yaml. (#2337, @ialidzhikov)
• [OPERATOR] The error reporting for shoot clusters has been improved. Configuration problems or similar issues which occur during shoot reconciliation are now instantly visible in the shoot resource. (#2336, @timuthy)
• [OPERATOR] Adjust default concurrent-sync settings for the Gardenlet controllers ShootState and ControllerInstallationRequired causing too many goroutines to be created. Also fixed a bug during worker creation to further reduce the amount of workers being created. (#2331, @danielfoehrKn)
• [OPERATOR] Fix the Seed enqueue in the federated seed controller for initially not-bootstrapped Seeds. Now Seeds are properly enqueued when their status changes from not-bootstrapped to successfully bootstrapped. (#2330, @danielfoehrKn)
• [OPERATOR] Update of the temporary, experimental Kyma addon to latest Kyma version 1.12.0. It can be installed onto shoot clusters out-of-the-box by annotating the Shoot with experimental.addons.shoot.gardener.cloud/kyma=enabled. Be aware that we won't provide upgrades or customization, and that this addon is temporary and will be removed in a future version of Gardener again. Its purpose is to ease the Kyma installation and to show-case which features it provides. It is by no means a production-ready setup. Also, please note that, once enabled, the Kyma addon can never be disabled again. The only way to get rid of it is to delete the shoot cluster. You can check the status of the installation by using kubectl get installation/kyma-installation -o jsonpath="{'Status: '}{.status.state}{', description: '}{.status.description}". (#2326, @a-thaler)
• [OPERATOR] An issue has been fixed with prevented failed shoots from being excluded from reconciliation after the retryDuration is exceeded. (#2324, @timuthy)
• [OPERATOR] Exclude logs from testing purpose or hibernated shoots (#2320, @vlvasilev)
• [OPERATOR] Removed safe-to-evict cluster-autoscaler annotation from etcd-events pods. (#2317, @georgekuruvillak)
• [OPERATOR] The kube-apiserver deployment of shoots is now scheduled with anti-affinity and a pod disruption budget of 1. (#2310, @rfranzke)
• [OPERATOR] An issue preventing gardenlet to reflect unhealthy Shoot conditions in the shoot.gardener.cloud/status label for newly created Shoots is now fixed. (#2308, @ialidzhikov)
• [OPERATOR] Fixed a bug in the health check library that leads to too many health checks being executed when the Extension.Status changes. (#2307, @danielfoehrKn)
• [OPERATOR] Fixed the health check condition.lastTransitionTime in the health check library. (#2307, @danielfoehrKn)
• [OPERATOR] The Gardenlet sets an additional annotation to Extension CRDs during reconciliation to guarantee an update event for the watching clients. (#2290, @danielfoehrKn)
• [OPERATOR] Fixes a bug in the extension libraries that could lead to duplicate reconciliation of extension resources. When respecting the operation annotation set by the Gardenlet during reconciliation, extension controllers now only watch the Extension CRD. (#2290, @danielfoehrKn)
• [OPERATOR] Fixes a bug in the extension library of all extension resources that lead to not stopping the reconciliation of extension resources when the Shoot is in 'failed' state (Shoot.Status.lastOperation.state = Failed). (#2279, @danielfoehrKn)
• [OPERATOR] An issue has been fixed that may have resulted in the deletion of extension controllers from a seed cluster although the seed still had extension objects that the extension controller was responsible for. (#2261, @rfranzke)
• [OPERATOR] Restrictions on kube-proxy are lifted to allow switching of kube-proxy mode (IPTables, IPVS) for k8s cluster > 1.16. (#2238, @DockToFuture)
• [OPERATOR] Seed lifecycle controller uses Lease object to report Seeds' readiness (#2151, @swilen-iwanow)
• [OPERATOR] Updating maintenance integration tests for minor Kubernetes updates. (#2109, @danielfoehrKn)
• [DEVELOPER] A utility function for cleaning up orphaned ClusterRoles for the machine-controller-manager was added to the extensions/pkg/controller/worker/genericactuator package. (#2378, @rfranzke)
• [DEVELOPER] The generic Worker actuator does now exit its reconciliation flows early if it detects an error during the machine reconciliation. This allows to faster propagate problems to the end-user. (#2348, @rfranzke)
• [DEVELOPER] The extension health check library does now allow individual health checks to return the Progressing status. This allows to provide more accurate status information and less false negative health reports. (#2289, @rfranzke)
• [DEVELOPER] It is now possible to add a global cleanup function for integration tests. (#2283, @schrodit)
• [DEVELOPER] It is now possible to add a dedicated AfterTest function to test cases to run a specific function when the test has finished. (#2283, @schrodit)
• [DEVELOPER] Extend the Extensions actuator interface with Migrate and Restore (#2277, @vlvasilev)

[etcd-druid]

Most notable changes

• [USER] Bump default etcd-backup-restore image version to v0.9.1. This fixes the false alerts for FullBackupFailed on etcd pod restart. ⚠️ Etcd pod will be restarted in next reconcile. (gardener/etcd-druid#56, @georgekuruvillak)
• [DEVELOPER] ⚠️ Etcd-druid NO MORE adds the annotation "cluster-autoscaler.kubernetes.io/safe-to-evict": "false" to etcd pods. Please make use of .spec.annotations to configure such annotation. (gardener/etcd-druid#55, @swapnilgm)

Docker Images

gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.5.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.5.0
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.5.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.5.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.5.0