karmada-io/karmada v1.4.0 on GitHub

What's New

Declarative Resource Interpreter

The Interpreter Framework is designed for interpreting the structure of arbitrary resource types. It consists of built-in and customized interpreters, this release introduced another brand-new customized interpreter.

With the newly introduced declarative interpreter, users can quickly customize resource interpreters for both Kubernetes resources and CRD resources by the rules declared in the ResourceInterpreterCustomization API specification. Compared with the interpreter customized by the webhook, it gets the rules from the declarative specifications instead of requiring an additional webhook component.

The new command named interpret in the karmadactl could be used to test the rules before applying them to the system.
Some examples are provided to help users better understand how this interpreter can be used.

(Feature contributor: @jameszhangyukun @ikaven1024 @chaunceyjiang @XiShanYongYe-Chang @RainbowMango)

PropagationPolicy/ClusterPropagationPolicy priority

Users are now able to declare the priorities for both PropagationPolicy and ClusterPropagationPolicy. A policy will be applied for the matched resource templates if there are no other policies with higher priority at the point the resource template be processed.

The system administrator can use priorities to manage and control policies. Refer to Configure PropagationPolicy priority for more details.

(Feature contributor: @Garrybest @jwcesign)

Instrumentation improvement

This release enhanced observability significantly through metrics and events.
The metrics can be queried by the endpoint(/metrics) of each component using an HTTP scrap, and they are served in Prometheus format. Events are reported to the relevant resource objects respectively.

Refer to events and metrics for more details.

(Feature contributor: @Poor12)

Other Notable Changes

API Changes

Introduces priority to PropagationPolicy. (#2758, @RainbowMango)
Introduced LabelsOverrider and AnnotationsOverrider for overriding labels and annotations.(#2584, @chaunceyjiang)
Evoluted PropagateDeps FeatureGate to Beta and enabled it by default. (#2875, @XiShanYongYe-Chang)
Evoluted Failover/GracefulEviction FeatureGate to Beta and enabled it by default. (#2876, @jwcesign)
Evoluted CustomizedClusterResourceModeling FeatureGate to Beta and enabled it by default. (#2877, @Poor12)

Bug Fixes

karmada-search: Fixed concurrent map writes panic while listing objects via proxy. (#2483, @ikaven1024)
karmada-search: Fixed the issue that the returned ResourceVersion by proxy was not stable. (#2746, @cmicat)
karmada-controller-manager/karmada-agent: Fixed the issue that pod information cannot be collected when building resource summary. (#2489, @Poor12)
karmada-controller-manager: Used cluster secret ref namespace in unified-auth-controller when generating a ClusterRoleBinding. (#2516, @XiShanYongYe-Chang)
karmada-controller-manager: Fixed the failure of creating resources whose name contains colons. (#2549, @XiShanYongYe-Chang)
karmada-controller-manager: Fixed the panic when cluster ImpersonatorSecretRef is nil. (#2675, @stingshen)
karmada-controller-manager: Fixed serviceaccount continual regeneration by service account controller. (#2578, @Poor12)
karmada-controller-manager: Disabled the preemption matching of pp/cpp priority. (#2734, @XiShanYongYe-Chang)
karmada-controller-manager: Fixed the issue that clusterOverridePolicy and overridePolicy with nil resource selector could not work. (#2771, @wuyingjun-lucky)
karmada-controlle-managerr: Ignored resources that do not match with a policy before applying the policy. (#2786, @XiShanYongYe-Chang)
karmada-agent: Fixed the issue that ServiceExport controller cannot report endpointSlices (due to the missing create permission). (#2515, @lonelyCZ)
karmadactl: Fixed init cannot honor IPv6 addresses when generating the kubeconfig file. (#2450, @duanmengkk)
karmadactl: Fixed --karmada-data directory was not initialized in init command. (#2548, @jwcesign)
karmadactl: Fixed the issue that init commands print incorrect register command. (#2707, @Songjoy)
karmadactl: Fixed the issue that the namespace already exists. (#2505, @cleverhu)
karmada-webhook: Fixed the failure to set resource selector default namespace when the relevant OverridePolicy and PropagationPolicy are not configured with namespaces. (#2858, @carlory)

Security

Security: Added limitReader to io.ReadAll which could limit the memory request and avoid DoS attacks. (#2765, @Poor12)

Features & Enhancements

karmadactl: Improved karmada init help output. (#2342, @my-git9)
karmadactl: karmadactl prohibits input extra arguments for init command. (#2497, @helen-frank)
karmadactl/chart: The init no longer creates redundant ServiceAccounts for components except karmada-agent. (#2523, @carlory)
karmadactl: Fixed the issue that options of deinit cannot be shown. (#2540, @helen-frank)
karmadactl/chart: If Karmada is installed by using karmadactl or a Helm chart with default configurations, the image tag of karmada-kube-controller-manager/karmada-apiserver will be kube-controller-manager:v1.25.2/kube-apiserver:v1.25.2. (#2539, @jwcesign)
karmadactl: Introduced --karmada-apiserver-advertise-address flag to specify Karmada APIserver's address to the init sub-command. (#2550, @wuyingjun-lucky)
karmadactl: Introduced --enable-cert-rotation option to register commands. (#2596, @lonelyCZ)
karmadactl: Added dryrun for uncordon. (#2760, @helen-frank)
karmadactl: Added validation on cluster existence for karmadactl get. (#2787, @helen-frank)
karmadactl: Added liveness probe into the kube-controller-manager component. (#2817, @carlory)
karmadactl: Added --image-registry flags for init. (#2655, @helen-frank)
karmadactl: Added interpreter command for resource interpretation customizations. (#2750, @ikaven1024)
karmadactl: Added execute mod for the interpret command. (#2824, @ikaven1024)
karmada-search: Objects returned by proxy will have resource.karmada.io/cached-from-cluster annotation to indicate which member cluster it comes from. (#2469, @ikaven1024)
karmada-search: Users can get the real resource request metrics while using the proxy. (#2481, @ikaven1024)
karmada-search: Users now can use --disable-search and --disable-proxy options to disable search and proxy features (both enabled by default). (#2650, @ikaven1024)
karmada-controller-manager: Added implicit priority for PropagationPolicy. (#2267, @Garrybest)
karmada-controller-manager: Introduced resource label namespace.karmada.io/skip-auto-propagation: "true" for namespaces that should be skipped from auto propagation. (#2696, @jwcesign)
karmada-controller-manager: Allowed users to update the .spec.resourceSelectors field of PropagationPolicy/ClusterPropagationPolicy. (#2562, @XiShanYongYe-Chang)
karmada-controller-manager`: Introduced priority to PropagationPolicy. (#2767, @jwcesign)
karmada-scheduler-estimator: Leveraged scheduler cache to estimate replicas. (#2704, @Garrybest)
karmada-controller-manager: Stop propagating finalizers to member clusters. (#2870, @stingshen)
karmada-scheduler/karmada-scheduler-descheduler: Introduced --scheduler-estimator-service-prefix flag for discovery estimators. (#2527, @carlory)
karmada-scheduler: Added scheduling diagnosis. (#2302, @Garrybest)
karmada-agent: Introduced auto certificate rotation function. (#2596, @lonelyCZ)
karmada-webhook: Prevented modifying and creating ResourceInterpreterCustomization using the same interpretation rules. (#2755, @chaunceyjiang)
Validate cluster fields: provider, region and zone. (#2849, @carlory)

Other

Dependencies

Download images from Docker hub by default. (#2795, @jwcesign)
Karmada is now built with Golang 1.19.3. (#2857, @RainbowMango)

Helm Chart

Helm Chart: Added descheduler name suffix of chart deployment manifest. (#2330, @calvin0327)
HelmChart: Fixed liveness probe misconfiguration which caused kube-controller-manager to always be CrashLoopBackup. (#2277, @calvin0327 )
chart: Fixed the issue that using custom certs leads to failed post-install-job and kube-controller-manager crash due to missing /etc/karmada/pki/server-ca.key. (#2637, @631068264)

Instrumentation

Instrumentation: Introduced the GetDependenciesSucceed and GetDependenciesFailed to resource templates. Introduced the SyncScheduleResultToDependenciesSucceed and SyncScheduleResultToDependenciesFailed to resourceBinding object. (#2773, @Poor12 )
Instrumentation: Introduced EvictWorkloadFromClusterSucceed and EvictWorkloadFromClusterFailed to the binding object and its reference. Refactored the event name of TaintManagerEviction. (#2835, @Poor12)
Instrumentation: Introduced metrics resource_find_matched_policy_duration_seconds, resource_apply_policy_duration_seconds, policy_apply_attempts_total, binding_sync_work_duration_seconds, and work_sync_workload_duration_seconds. (#2868, @Poor12 )
Instrumentation: Introduced the CreateExecutionSpaceSucceed and RemoveExecutionSpaceSucceed events to Cluster object. (#2688, @Poor12)
Instrumentation: Introduced the ApplyOverridePolicySucceed and ApplyOverridePolicyFailed events to workloads. (#2764, @Poor12)
Instrumentation: Introduced the ReflectStatusToWorkSucceed, ReflectStatusToWorkFailed, InterpretHealthSucceed and InterpretHealthFailed events to work object. (#2770, @Poor12)
Instrumentation: Introduced the SyncImpersonationConfigSucceed and SyncImpersonationConfigFailed to the cluster object. (#2796, @Poor12)
Instrumentation: Applied AggregateStatusFailed, AggregateStatusSucceed, SyncWorkSucceed, and SyncWorkFailed to federatedResourceQuota object. (#2812, @Poor12)
Instrumentation: Introduced SyncDerivedServiceSucceed and SyncDerivedServiceFailed to serviceImport object. (#2830, @Poor12)
Instrumentation: Introduced the cluster_ready_info, cluster_node_number, cluster_ready_node_number, cluster_memory_allocatable_bytes, cluster_cpu_allocatable_number, cluster_pod_allocatable_number, cluster_memory_allocated_bytes, cluster_cpu_allocated_number, cluster_pod_allocated_number, cluster_sync_status_duration to record the cluster status in karmada-controller-manager and karmada-agent. (#2496, @Poor12)
Instrumentation: Introduced metrics framework_extension_point_duration_seconds and plugin_execution_duration_seconds for karmada-scheduler. (#2087, @Poor12)

Contributors

Thank you to everyone who contributed to this release!

Users whose commits are in this release (alphabetically by username)

karmada-io/karmada v1.4.0 karmada v1.4.0 release on GitHub