[gardener]
Action Required
- [USER] If
APIServerSNI
feature gate is enabled by the Gardener administrators, a TLS client with support for server name indication must be used when talking to Shoot API servers. Alternatively,unmanaged
DNS provider should be used. (#2406, @mvladev) - [DEVELOPER] The creation and usage of kubernetes clientsets in controllers across gardener/gardener has been refactored. Please use the newly introduced
ClientMap
s to retrieve clientsets for all kind of clusters instead of creating new clientsets each time to leverage the clients' caches. (#2449, @tim-ebert) - [DEVELOPER] Kubernetes clientsets (
pkg/client/kubernetes.Interface
) now return a cached controller-runtime client fromInterface.Client()
ifkubernetes.UseCachedRuntimeClients
has been set to true (defaults tofalse
). The cached clients have to be started before the first usage by a call toInterface.Start()
. If you use aClientMap
to retrieve the clientset, this is done automatically. (#2449, @tim-ebert)
Most notable changes
- [USER] Fixed a bug where for Shoot clusters >= 1.17,
kube-apiserver
did not have any root CA bundles. This resulted in failure to verify x509 certificates when attempting to send traffic to OIDC discovery endpoint or other endpoints. (#2508, @mvladev) - [USER] The upstream community
vertical-pod-autoscaler
component is now supported for shoot clusters by setting.spec.kubernetes.verticalPodAutoscaler.enabled=true
(disabled by default). More information can be found in this document. (#2478, @rfranzke) - [USER] It is now correctly advertised in the OpenAPI specification that the
name
property fordataVolumes
in theShoot
spec is a required field. (#2463, @rfranzke) - [OPERATOR] A bug which blocked APIserver deployments on the same node (due to hostPort usage) is now fixed. (#2477, @zanetworker)
- [OPERATOR] A new FeatureGate
CachedRuntimeClients
has been added togardenlet
,gardener-controller-manager
andgardener-scheduler
. If enabled via the respective component config, the components use cached clients for their API calls wherever possible. (#2449, @tim-ebert) - [OPERATOR] The gardener-controller-manager does now automatically delete stale
Project
s that are no longer in use. By default, onlyProject
s older than30d
which are unused for at least14d
will be auto-deleted after90d
. However, the concrete values depend on the configuration of the respective Gardener landscape. You can find more information in this document. (#2446, @rfranzke) - [OPERATOR] Enabling the feature gate
APIServerSNI
can cause thekube-controller-manager
to be scaled down to0
for15 minutes
. This is a known issue and it's going to be resolved in a future release ofdependency-watchdog
. (#2406, @mvladev) - [OPERATOR] The feature gate
APIServerSNI
implementing GEP-8 is now available at alpha state. This allows for only one LoadBalancer in a Seed cluster to be used for all Shoot clusters in it. It's recommended to use in conjunction withManagedIstio
feature gate as the feature requires Istio to be installed in the Seed cluster. (#2406, @mvladev) - [DEVELOPER] The Terraformer can now deal with output types other than String in the Terraform state. (#2460, @timuthy)
Improvements
- [USER] It is now possible to restrict core components from running on a worker pool by specifying
systemComponents.allow: false
in the pool definition. (#2480, @jannickfahlbusch)- System components deployed by extensions (Such as typha from calico) need a separate adaptation and are not covered by this change.
- [USER] The
.spec.maintenance
settings are now correctly defaulted when aShoot
is being created without any such configuration. (#2464, @rfranzke) - [OPERATOR] Added vpa for hvpa-controller (#2553, @ggaurav10)
- [OPERATOR] Fixed possible race condition when updating the ShootState. (#2543, @plkokanov)
- [OPERATOR] An issue has been fixed, which caused the deletion of hibernated Shoots to be blocked if the
KonnectivityTunnel
has been enabled while the Shoot was hibernated. (#2540, @tim-ebert) - [OPERATOR] Failed shoot conditions are set to
Progressing
status for the configuredconditionThresholds
time after a successful shoot reconciliation. This is to prevent false negative status reports shortly after reconciliations. (#2535, @rfranzke) - [OPERATOR] Fixes a bug that could lead to defaulting a machine image of a Shoot to a preview version. (#2534, @danielfoehrKn)
- [OPERATOR] Conditions are now not only pardoned for
Create
/Delete
operations but also for processingReconcile
operations in case there aren't any last errors. (#2533, @rfranzke) - [OPERATOR] It is now possible to configure the enabled
FeatureGates
for thegardener-scheduler
via the respective values in thegardener/controlplane
chart. (#2531, @tim-ebert) - [OPERATOR] The istiod validating webhook on Seeds is now exposed at port
443
, allowing it to function properly in GKE clusters. (#2529, @tim-ebert) - [OPERATOR] Adapts values for hvpa's LimitsRequestsGapScaleParams to latest hvpa-controller version (#2521, @ggaurav10)
- [OPERATOR] Backupentry is now properly ignored when trying to annotate it with gardener.cloud/operation=migrate if it has already been deleted from the cluster. (#2520, @plkokanov)
- [OPERATOR] An issue has been fixed which caused Gardener to delete on-demand extensions prematurely. (#2517, @timuthy)
- [OPERATOR] Removed the generic tolerations for all taints from control-plane component deployments. This is required for dedicated worker pool nodes to host only ETCD pods if gardener/kupid extension is deployed in the seed clusters. (#2507, @amshuman-kr)
- [OPERATOR] Fixed a bug, that caused
gardener-controller-manager
andgardenlet
to panic if the Kubeconfig referenced by a Plant or Seed is empty. (#2504, @tim-ebert) - [OPERATOR] Fixed a bug that leads to Shoots not receiving a force minor version update when the Kubernetes AutoUpdate is enabled. (#2490, @danielfoehrKn)
- [OPERATOR] Added the metrics
shoot:container_network_transmit_bytes_total_apiserver:sum
andshoot:container_network_receive_bytes_total_apiserver:sum
which will be useful in observing the network traffic for all shoots. (#2488, @wyb1) - [OPERATOR]
ManagedIstio
is updated to1.6.3
(#2487, @mvladev) - [OPERATOR] A bug has been fixed that prevented the HPA for istio to work as expected when the
ManagedIstio
feature gate was enabled. (#2486, @rfranzke) - [OPERATOR] Konnectivity tunnel is now updated to v0.0.10. (#2484, @zanetworker)
- [OPERATOR] Runtime metrics for Pods and Nodes are now also available in environments which don't support a domain name resolution for worker nodes. (#2468, @timuthy)
- [OPERATOR] An issue has been fixed which prevented the
retry
operation for shoots from working reliably in case of a reconciliation. (#2467, @timuthy) - [OPERATOR] CoreDNS pods are now protected by a PDB during machine upgrades and should reside on different nodes for HA. (#2466, @zanetworker)
- [OPERATOR] ControllerInstallations are not removed from the Seed if there is at least one Shoot referring it in the
spec.seedName
orstatus.seedName
. (#2456, @swilen-iwanow) - [OPERATOR] It is now possible to keep a
Namespace
in the system even when the relatedProject
is deleted by annotating theNamespace
withnamespace.gardener.cloud/keep-after-project-deletion=true
. (#2436, @rfranzke) - [OPERATOR] Secrets deployed by gardener in the
Shoot
's Control Plane are now saved/loaded to/from the ShootState. (#2359, @plkokanov) - [DEVELOPER] Docker images built by
make docker-images
are now tagged and build with the commit hash appended to the version. (#2500, @tim-ebert) - [DEVELOPER] ChartApplier's
Delete
and ManifestReader'sDeleteManifest
now support passingTolerateErrorFunc
option which can be used to tolerate certain errors - e.g. usingTolerateNoMatchError
can be useful in situations where a deleting a custom resource, but its CRD is already removed. (#2496, @mvladev) - [DEVELOPER] General information about Gardener Enhancement Proposals (GEPs) have been added. Please consult this documentation for more information. (#2479, @timuthy)
- [DEVELOPER] Minor fixes for hook-me.sh script (#2473, @prashanth26)
- [DEVELOPER] The
EnsureCleanedUp
andWaitForCleanEnvironment
funcs are now exported via theTerraformer
interface. (#2461, @tim-ebert)
[dependency-watchdog]
Improvements
- [OPERATOR] Minimize throttling for happy path of probes (when targets do not need to be updated). (gardener/dependency-watchdog#20, @amshuman-kr)
- For example, load the current replicas via the local cache, add jitter to the probe intervals to spread out host apiserver call, make client-go QPS and Burst configurable via CLI flags and export load-related metrics.
[gardener-resource-manager]
Most notable changes
- [OPERATOR] Please ensure, that
gardener-resource-manager
has the required permissions to also update secrets now. (gardener/gardener-resource-manager#54, @tim-ebert)
Improvements
- [USER] All resources managed by
resource-manager
now haveresource-manager.gardener.cloud/description
annotation withDO NOT EDIT
warning. (gardener/gardener-resource-manager#46, @mvladev) - [OPERATOR] Missing RBAC rules for updating and patching secrets were added. (gardener/gardener-resource-manager#71, @tim-ebert)
- [OPERATOR] A bug has been fixed, that caused new Services to get assigned a different ClusterIP than specified. (gardener/gardener-resource-manager#68, @tim-ebert)
- [OPERATOR] Fixed a bug, that caused the deletion of Services to be blocked. (gardener/gardener-resource-manager#65, @tim-ebert)
- [OPERATOR] Fixed a bug, that caused the deletion of RBAC resources to be blocked. (gardener/gardener-resource-manager#63, @tim-ebert)
- [OPERATOR] A forceful reconciliation for managed resources can now be triggered by the annotation
gardener.cloud/operation: reconcile
. (gardener/gardener-resource-manager#57, @timuthy) - [OPERATOR] The Gardener-Resource-Manager does now try to recover unhealthy managed resources (condition
ResourcesHealthy
) by reconciling the resource. (gardener/gardener-resource-manager#57, @timuthy) - [OPERATOR]
gardener-resource-manager
now properly removes its finalizer from secrets, that are not referenced by aManagedResource
anymore. (gardener/gardener-resource-manager#54, @tim-ebert) - [OPERATOR] A bug has been fixed, which could lead to a situation, where a
ManagedResource
is falsely indicating a "Ready" state for a short period of time. (gardener/gardener-resource-manager#51, @tim-ebert) - [OPERATOR] The
ManagedResource
CRD features a new field.spec.deletePersistentVolumeClaims
. If set totrue
, gardener-resource-manager will delete PVCs belonging to managed StatefulSets, when they are deleted. (gardener/gardener-resource-manager#50, @tim-ebert) - [OPERATOR]
gardener-resource-manager
now also injects labels specified in.spec.injectLabels
into the.spec.volumeClaimTemplates
of new StatefulSets. (gardener/gardener-resource-manager#49, @tim-ebert) - [OPERATOR]
gardener-resource-manager
now deletes resources withDeletePropagationForeground
to cascade the deletion to their dependents (e.g. to clean upJob
s created by aCronJob
). (gardener/gardener-resource-manager#48, @tim-ebert) - [OPERATOR] The logs of
gardener-resource-manager
have been reworked to contain less unnecessary error entries. (gardener/gardener-resource-manager#45, @tim-ebert) - [OPERATOR]
gardener-resource-manager
now keeps the status of managed objects to prevent overwriting the status of CRs that don't have astatus
subresource. (gardener/gardener-resource-manager#44, @tim-ebert) - [OPERATOR]
gardener-resource-manager
now keeps the replicas and/or resource requirements of Deployments and StatefulSets if they are scaled horizontally and/or vertically by an HPA or HVPA respectively. (gardener/gardener-resource-manager#44, @tim-ebert) - [OPERATOR] The
ResourcesApplied
condition ofManagedResource
s now includes all errors, that occurred while applying/deleting managed objects if there were any. (gardener/gardener-resource-manager#43, @tim-ebert) - [OPERATOR] A bug has been fixed, which made
gardener-resource-manager
fail to apply all new objects, if there were conflicting changes on those objects, instead of retrying the update request. (gardener/gardener-resource-manager#42, @tim-ebert) - [OPERATOR]
gardener-resource-manager
now adds finalizers to Secrets referenced inManagedResource
s to prevent Secrets from being deleted accidentally. (gardener/gardener-resource-manager#41, @tim-ebert) - [OPERATOR]
gardener-resource-manager
now makes use of a caching client for talking to the targeted API server, which reduces its network traffic. (gardener/gardener-resource-manager#40, @tim-ebert) - [OPERATOR]
gardener-resource-manager
handling for Jobs is now improved. (gardener/gardener-resource-manager#37, @ialidzhikov) - [OPERATOR] Service merge now handles headless
ClusterIP
services andExternalName
services..spec.healthCheckNodePort
is only set if the service is of typeLoadBalancer
with.spec.externalTrafficPolicy: Local
(gardener/gardener-resource-manager#35, @mvladev) - [OPERATOR] A bug has been fixed, that caused new Services to get assigned a different ClusterIP than specified. (gardener/gardener-resource-manager@276bf6d)
- [OPERATOR]
gardener-resource-manager
handling for Jobs is now improved. (gardener/gardener-resource-manager@6e40fe5) - [DEVELOPER] The new
--always-update
command line parameter (default:false
) allows to configure whether to always send aPUT
request for managed resources regardless of whether their desired state differs from their actual state. (gardener/gardener-resource-manager#70, @rfranzke)
[hvpa-controller]
Improvements
- [OPERATOR] Minor bug fix: Use vpa scale policies correctly (gardener/hvpa-controller#73, @ggaurav10)
- [OPERATOR] Change handling of LimitsRequestsGapScaleParams: Use
max
of value and percentage gaps, instead ofmin
(gardener/hvpa-controller#72, @ggaurav10) - [OPERATOR] Now HVPA doesn't take VPA recommendations into account if VPA condition has
ConfigUnsupported
,ConfigDeprecated
orLowConfidence
set totrue
(gardener/hvpa-controller#68, @ggaurav10) - [OPERATOR] Removing "Temporary/fast fix to enable scale down even if vpaWeight == 0" as we have better ways to optimise cost now (gardener/hvpa-controller#64, @ggaurav10)
- [OPERATOR] Ignore
minChange
configuration while overriding scale up stabilisation. This ensures that full VPA recommendations are applied in case the target pods are OOMKilled or restarted due to livenessProbe failure, no matter what. (gardener/hvpa-controller#61, @amshuman-kr) - [OPERATOR] Consider HPA to be limited if we have seen oomkill or liveness probe fails already. This change makes HVPA controller scale the app vertically more actively, ignoring the HPA's status condition type
ScalingLimited
. (gardener/hvpa-controller#57, @ggaurav10) - [OPERATOR] Consider HPA scale out to be limited in case, overrideScaleUpStabilization is set in the status and hpa weight is 0 so that full VPA recommendation is immediately applied. (gardener/hvpa-controller#56, @ggaurav10)
- [OPERATOR] Add ci master build status and go report card badges (gardener/hvpa-controller#54, @ggaurav10)
Docker Images
gardener-apiserver: eu.gcr.io/gardener-project/gardener/apiserver:v1.7.0
gardener-controller-manager: eu.gcr.io/gardener-project/gardener/controller-manager:v1.7.0
gardener-scheduler: eu.gcr.io/gardener-project/gardener/scheduler:v1.7.0
seed-admission-controller: eu.gcr.io/gardener-project/gardener/seed-admission-controller:v1.7.0
gardenlet: eu.gcr.io/gardener-project/gardener/gardenlet:v1.7.0