Cluster API v1.2 - Release notes

This release is all about extensibility and adding more capabilities to ClusterClass and managed topologies.

By using the new Runtime SDK and Lifecycle Hooks it is now possible to build systems on top of Cluster API by hooking into a workload cluster’s lifecycle.

By using the Topology Mutation Hook (also built on top of Runtime SDK) it is now possible to write ClusterClass patches using the programming language of your choice, thus allowing to support more sophisticated Cluster variants with a single ClusterClass.

Also, starting from this release, ClusterClass and Managed Topology is using Server Side Apply, thus providing better support - fully aligned with the Kubernetes ecosystem - for use cases where different controllers co-author the same object while provisioning Clusters.

And as usual, many many other brilliant improvements and bug fixes from this amazing community!

Thanks to all our contributors! 😊

Changes since v1.1.5

👌 Kubernetes version support

• Management Cluster: v1.20.x -> v1.24.x [1]
• Workload Cluster: v1.18.x -> v1.24.x

[1] When using CAPI with the CLUSTER_TOPOLOGY experimental feature on, the Kubernetes Version for the management cluster must be >= 1.22.0.

⚠️ Breaking Changes

• Bump min management cluster version to 1.20.2 (#6597)
• Make ClusterToInfrastructureMapFunc check if the cluster is externally managed (#6070)

📝 Proposals

• ClusterClass: Add MD variables to the ClusterClass proposal (#5976)
• Runtime SDK: Add Runtime SDK proposal (#6181)
• Runtime SDK: Add Runtime Hooks for Add-on management proposal (#6418)
• Runtime SDK: Add Topology Mutation Hook proposal (#6349)
• MachinePool: Add proposal for MachinePool Machines (#6088)
• IPAM: Add IPAM integration proposal (#6000)
• Metrics: Add Cluster API State Metrics proposal (#6404)

And #6810, #6743, #6568, #6822, #6234, #5994

✨ New Features

• CABPK: Add support for kubeadm skipPhases field (#5993)
• CABPK: Add support for passwd from secret (#6386)
• CABPK: Refresh BootstrapToken until Nodes join (#6395)
• CABPK: Expose kubeadm bootstrap controller bootstrap token ttl config (#6584)
• ClusterClass: Apply PodSecurityStandard in Secure Cluster Class (#6390)
• clusterctl: Migrate CRDs during clusterctl upgrade (#6793)
• clusterctl: Upgrade apply: wait for providers to be ready (#6685)
• clusterctl: Prebuilt container for clusterctl in image based pipelines (#6405)
• clusterctl: Add options to show templates and cluster resource sets (#5762)
• clusterctl: Allow Namespace objects in topology plan input (#6257)
• Developer Experience: introduce runtime-openapi-gen to generate openapi spec on release (#6462)
• E2E: Allow provider specific infra machine template for upgrade tests (#6075)
• IPAM: Add IPAddress and IPAddressClaim CRs to Experimental API (#6887)
• KCP: Make KCP's patches option mutable (#6586)
• KCP: Make SetNodeProviderID no more blocking although Control Plane is externally managed (#6640)
• Logging: Add JSON log format and deprecate klog flags (#6072)
• Machine: Add nodeDeletionTimeout property to Machine (#5608)
• Runtime SDK: Implement BeforeClusterDelete hook (#6802)
• Runtime SDK: Implement AfterControlPlaneInitialized, AfterControlPlaneUpgrade and AfterClusterUpgrade hooks (#6629)
• Runtime SDK: Topology Mutation Hook: Implement external patching (#6623)
• Runtime SDK: BeforeClusterCreate, BeforeClusterUpgrade implementation (#6608)
• Runtime SDK: Add caBundle injection to Extension controller (#6632)
• Runtime SDK: Add Test Extension and initial e2e test (#6625)
• Runtime SDK: Topology Mutation: use API types for inline patching (#6289)
• Runtime SDK: Implement Call and CallAll in RuntimeClient (#6499)
• Runtime SDK: Topology Mutation: Add types for Topology Mutation Hook (#6514)
• Runtime SDK: Add client and Discovery (#6455)
• Runtime SDK: Add registry (#6449)
• Runtime SDK: Add extension config controller skeleton (#6411)
• Runtime SDK: Implement Catalog for Runtime SDK (#6393)
• Runtime SDK: Add API types for Runtime SDK ExtensionConfig (#6383)
• SSA: Add Server Side Apply helper to the topology controller (#6495)
• SSA: Improve dry run for topology changes to dry run server side apply (#6861)
• SSA: Add PatchAndWait helper function for server-side apply (#6572)
• Testing: Introduce typed CRDs for testing purposes to replace the untyped ones (#6573)
• Tilt: Add prometheus to observability deployment (#6117)

And #6777, #6487,, #6326, #6457, #6475, #6494, #6367, #6348, #6292, #6335, #6140, #6124, #6082, #5999

🐛 Bug Fixes

• API: Fix panics in conversions (#6144)
• Book: Fix cert manager repo and add troubleshooting guide (#6050)
• CAPD: Update docker distribution to fix go sum error (#6548)
• CAPD: Add containerd socket to worker nodes (#6473)
• CAPD: Update docker distribution dependency version (#6199)
• CAPD: Use Kubernetes 1.24 in quickstart and CAPD, bump to kind v0.14 (#6513)
• CAPD : Exclude externally managed infrastructures in ClusterToInfrastructureMapFunc
• CABPK: Add Ignition base64 and file ownership support, reject gzipped content and fix unit tests (#6472)
• Cluster: Fix reconcilation predicates (#6425)
• ClusterClass: Delete unused ControlPlane InfrastructureMachineTemplates on rotation (#6399)
• ClusterClass: Reconcile InfrastructureCluster controlPlaneEndpoint (#6673)
• ClusterClass: Remove generated names from error messages to reduce reconciliation (#5971)
• ClusterClass: Set apiVersion to avoid diff in patchHelper (#6295)
• clusterctl: Retry github i/o operations (#6430)
• clusterctl: Retry delete during clusterctl upgrade (#6266)
• clusterctl: Make listObjByGVK resilient (#6262)
• clusterctl: Skip templating step if describe flag is used (#6347)
• clusterctl: Fix the version assumption in topology plan command (#6270)
• ClusterResourceSet: Handle error in case ownerRef is not set on crs binding (#5969)
• ClusterResourceSet: Apply objects in sorted order (#6502)
• Developer Experience: Work around lack of kubebuilder-tools for darwin/arm64 (#6092)
• Developer Experience: Cloudbuild fixes: (#6585, #6627, #6631)
• Developer Experience: Add workaround to run generate-go-openapi in GOPATH (#6607)
• E2E Framework: fix node drain output (#6174)
• E2E: fix cgroup branching in cluster upgrade test (#6557)
• E2E: use KUBERNETES_VERSION in kubeadm script (#6847)
• E2E: Update CoreDNS tags and upgrade checks in e2e tests (#6339)
• E2E: update clusterctl version in e2e to latest patch (#6057)
• E2E: test/e2e add tag to e2e tests which use ClusterClass (#6605)
• E2E: Retry Apply workloadClusterTemplate to fix flake (#6791)
• etcd: Bump go package to v3.5.3 (#6414)
• etcd: Close etcd leaky connection if client is not initialized properly (#6540)
• KCP: Don't rollout machines when format is defaulted (#6096)
• KCP: Allow KCP to Update when CoreDNS version doesn't change (#5986)
• KCP: Ensure controlplane coredns update deploys the ClusterRole if CoreDNS was already updated (#6759)
• KCP, CAPD: prevent blocking of KCP and DockerMachine controllers (#6183)
• KCP, CAPBK: Fix version comparison for pre release versions (#6649)
• Logging: Fixed formatting in runtime client (#6922)
• Machine: Fix panic in node drain (#6200)
• Machine: Improve to get error stack trace for machine reconcile (#6746)
• MachinePool: Ensure MachinePool Version is a valid Kubernetes version (#6311)
• MachinePool: Use APIReader to fix flaky MachinePool test (#6637)
• MachinePool: Use WaitForMachinePools to configure machine pool test waits (#6446))
• MachinePool: Permit empty/not set providerIDList (#6233)
• MachinePool: No nodeRefs when empty providerIDList is ok (#6312)
• MachineSet: Consistent ordering for deletion priority (#6300)
• MHC: Use .spec.clusterName instead of .metadata.clusterName for external remediation request (#6306)
• MHC: Stop adding parent object owner reference for MachineHealthChecks in managed topologies (#6660)
• MHC: Stop adding parent object owner reference in managed topologies (#6660)
• RuntimeSDK: Include ExtensionConfig CRD in generated core manifest (#6456)
• SSA: Create cluster-shim with regular client, use ignorePaths on create (#6915)
• SSA: Ignore diff of other managers (#6888)
• SSA: Carry over metadata.uid at ServerSidePatchHelper (#6784)
• SSA: Adjust structuredmerge patch helper options to set correct allow list for Cluster objects to prevent co-ownership (#6776)
• SSA: Fix structuredmerge filterintent to cleanup nested empty maps (#6796)
• SSA: Use the Kubernetes decoder in the patch engine (#6668)
• Testing: Fix flaky Test_objectMover (#6046)
• Testing: inline builders in test for MHC reconcilation (#5987)
• Tilt: Deploy observability tools in correct namespace (#6078)
• Tilt: Support using kind cluster from tilt settings automatically (#6244)
• Tilt: Support for newer tilt versions (#6630)
• Tilt: Fix infinite reload issue (#6187)
• Utils: Add option to compare without prereleases (#6252)
• Utils: Add nil pointer check in UnstructuredUnmarshalField (#6334)
• Utils: Fix nil pointers in conditions patch utils (#6813)

And #5977, #6638, #6028, #6231, #6707, #6351, #6682, #6731

🌱 Others

• Add killianmuldoon to bootstrap & docs reviewers (#6250)
• Add sbueringer to Cluster API maintainers (#6109)
• Add ykakarap to clusterctl reviewers (#6275)
• Build: Restrict permissions of GitHub actions (#6824)
• Build: Consolidate test/infrastructure/docker/Makefile to Makefile (#6443)
• Build: hack/release-notes: trim [release-*] in PR titles (#6574)
• Build: Install hack/tools with temporary go module (#5741)
• Build: Silence error in go_install.sh when trying to remove nonexisting files (#6532)
• Build: Split core conversion Makefile targets (#6498)
• Build: Update promote-images to filter by image and sort by tag (#6452)
• Build: Use quiet mode on markdown link checker (#6167)
• Build: Restructure makefile (#6132)
• CABPK: Deprecate experimentalRetryJoin (#6941)
• CAPD: Add docker mount to worker nodes (#6460)
• CAPD: Prefix criSocket path with unix:// in test configuration YAMLs (#6169)
• CI: Included Github Actions in the dependabot config (#6340)
• ClusterCacheTracker: Drop unused Log field (#6318)
• ClusterClass: Implement map variables (#6369)
• ClusterClass: Add topology field for NodeDrainTimeout (#6379)
• ClusterClass: Add ref and controlPlane name builtin variables (#6208)
• ClusterClass: Add network builtins to topology controller (#6171)
• ClusterClass: Add sprig support for variable templates (#6131)
• ClusterClass: Migrate reconcile_state tests to envtest (#6491)
• ClusterClass: Use name index in webhook (#5810)
• clusterctl: Add hetzner provider to the clusterctl and to the docs (#6062)
• clusterctl: Always use -n as the shorthand for --[target-]namespace (#6258)
• clusterctl: Remove old aws-eks providers (#6344)
• clusterctl: Make clusterctl init idempotent (#6474)
• clusterctl: Support disable version check (#6439)
• clusterctl: Replace uitable with tablewriter in clusterctl describe cluster command (#5942)
• CRDs: Order replica columns in a consistent order (#6294)
• CRDs: Add desired number column to KCP, MD, MS, MachinePool (#6164)
• E2E: Stream runtime SDK extension controller logs to artifacts (#6902)
• E2E: Add PostMachinesProvisioned to ApplyClusterTemplateAndWaitInput (#6866)
• E2E: Pull non-existent images when building kind bootstrap cluster (#6821)
• E2E: Remove CI image pull and tagging for kubeadm injection script (#6590)
• E2E: Retryable errors in test/framework (#6485)
• E2E: More retryable errors in test/framework (#6490)
• E2E: Tolerate zero pods in WaitForPodListCondition (#6478)
• E2E: Actually retry controller-runtime client instantiation (#6437)
• E2E: Retry controller-runtime client instantiation (#6431)
• E2E: Checks unexpected rollouts during clusterctl upgrade (#6184)
• E2E: Add option to configure both control-plane and workers machine template for k8s upgrade tests (#6201)
• E2E: Use env value to set kustomize path in test (#6205)
• E2E: Log if a new cluster is created or an existing is used (#6217)
• KCP: Improve validation webhooks (#6129)
• KCP: Handle kubeadm 1.24 kubelet ConfigMap name change (#6176)
• linter: enable noctx and unused (#5959)
• Logging: Set correct klog format (#6190)
• Logging: Log to logger from context during drain (#6170)
• Logging: Use klog.Background (#6602)
• Logging: log-push: preserve original timestamp, parse controller too (#6260)
• Logging: log-push: add tool to push logs to Loki (#6189)
• Make MachinesByCreationTimestamp private to machine collections (#6111)
• Makefile: re-enable hack/tools on make help (#6441)
• Remove code deprecated with v1.1.0 (#6114)
• Remove deprecated webhook code (#6225)
• Runtime SDK: Add logs (#6889)
• Runtime SDK: Add BeforeClusterDelete to runtimeSDK e2e tests (#6849)
• Runtime SDK: Better error handling for tracking utilities (#6811)
• Runtime SDK: Add blocking end-to-end tests for lifecycle hooks (#6800)
• Runtime SDK: Add metrics to Runtime SDK hook client (#6755)
• Runtime SDK: Export util packages (#6754)
• Runtime SDK: Improve error wrapping and godoc (#6752)
• Runtime SDK: Initial e2e test for Runtime SDK lifecycle hook (#6702)
• Runtime SDK: test/extension: add first version of server lib & topology mutation (#6635)
• Runtime SDK: Add index for ExtensionConfigs having ca injection annotation set (#6648)
• Runtime SDK: Enforce https for extensions (#6645)
• Runtime SDK: Implement namespace selector (#6619)
• Runtime SDK: Add client defaulting and validation for DiscoveryResponse (#6578)
• Runtime SDK: Implement aggregateResponse for Runtime client (#6581)
• Runtime SDK: Change webhook default port to 443 (#6580)
• Runtime SDK: Add lifecycle hook types (#6537)
• Runtime SDK: Standardize import names (#6561)
• Runtime SDK: Integrate controller for ExtensionConfig (#6510)
• Runtime SDK: Change ExtensionConfig to Cluster scoped (#6547)
• SSA: Improve comment about metadata.uid (#6912)
• Support new control plane label and taint (#5919)
• Testing: Add deepcopy methods to remaining builders (#6771 #6757 #6751 #6750)
• Testing: Use internal/builders in bootstrap (#5476)
• Tilt: Switch to YAML for configuration (#6133)
• Tilt: Update kubectl used for CAPD (#6121)
• Update Dockerfile syntax to v1.4 (#6659)
• Upgrade cluster-bootstrap, cobra, viper, grpc, .. (#6689)
• Upgrade cobra and viper (#6287)
• Upgrade Docker and containerd dependencies (#5972)
• Upgrade to controller-runtime v0.12.3 and k8s.io/* v0.24.* (#6359, #6467 #6724, #6848)
• Upgrade to controller-tools v0.9.1 (#6766)
• Upgrade to cert-manager v1.8.2 (#6420 #6744)
• Upgrade to corefile-migration v1.0.17 (#6389 #6525 #6600)
• Upgrade to docker v20.10.16 (#6552)
• Upgrade to go-github v45.2.0 (#6723)
• Upgrade to go 1.18.3 for Tiltfile, Container Images, Netlify (#6616)
• Upgrade to golangci-lint v1.45.2 (#6014, #6335)
• Upgrade to kind v0.14.0 (#6280 #6513)
• Upgrade to kubebuilder binaries v1.24.1 (#6013 #6604)
• Upgrade to kustomize v4.5.2 (#6213)
• Upgrade to actions/cache action v3.0.2 (#6355 #6409)
• Upgrade to actions/checkout action v3 (#6357)
• Upgrade to actions/setup-go action v3 (#6408)
• Upgrade to EndBug/add-and-commit action v9 (#6354)
• Upgrade to golangci-lint action v3.2 (#6361 #6524)
• Upgrade to kubebuilder-release-tools action v0.2.0 (#6492)

And #6898, #6897, #6883, #6870, #6891, #6823, #6763, #6748, #6696, #6665, #6662, #6658, #6643, #6603, #6562, #6453, #6436, #6406, #6377, #6376, #6368, #6363, #6352, #6358, #6322, #6214, #6207, #6159, #6193, #6180, #6107, #6079, #6125, #6099, #6097, #6032, #6002, #5974, #5964, #5982

📖 Additionally, there have been the following contributions to our documentation and book. (#6919, #6909, #6911, #6910, #6882, #6874, #6852, #6851, #6829, #6814, #6812, #6798, #6797, #6747, #6740, #6730, #6719, #6718, #6692, #6690, #6624, #6521, #6282, #5558, #6543, #6591, #6587, #6565, #6567, #6569, #6564, #6553, #6550, #6538, #6541, #6530, #6422, #6463, #6451, #6435, #6426, #6424, #6392, #6419, #6325, #6307, #6298, #6296, #6177, #6263, #6288, #6256, #6232, #6235, #6224, #6221, #6168, #6192, #6166, #6122, #6137, #6112, #6105, #6116, #6100, #6104, #6102, #6108, #6067, #6090, #6026, #6033, #6038, #6047, #5989, #5978, #6018, #5997, #6009, #5998, #5980, #6421, #6342, #6496, #6328, #6309, #6021, #6727, #6212, #6179, #6162, #6063, #5979)