Cluster API v1.3 Release Notes

Highlights

This release was focused on improving existing features, stability and our release process,
and as usual tons of hard work with more than 400 PRs from about 80 awesome contributors.

Some highlights:

• During the v1.3 release cycle we established a release timeline & release team and corresponding processes and documentation. This is a big step forward to make Cluster API more predictable and easier to use. Huge kudos to all volunteers for our first release team that will start with the Cluster API v1.4 release cycle.
  • v1.4 Release timeline & team (tentative release date for v1.4.0 is 28th March 2023)
  • Release team, Release cycle and Release tasks
• Cluster API can now automatically renew the machine certificates on machines provisioned by Kubeadm Control Plane (KCP) provider solving another management paint point for workload clusters (documentation).
• Cluster API images are now published and consumed from the new registry.k8s.io Kubernetes registry. Additionally, KCP now also automatically uses the new registry as default, starting with Kubernetes v1.22. See the Kubernetes blog post for more information.
• It's now possible to create Clusters without taints on control plane nodes. This makes it easy to schedule workloads on control plane nodes. This is useful for example for single node clusters.
• clusterctl can now manage IPAM and RuntimeExtension providers.
• Various improvements to ClusterClass and Runtime SDK, e.g.:
  • We added fields to allow configuration of MHC, MachineDeployment Strategy and many other fields to Cluster.spec.topology (API documentation).
  • For Runtime SDK authors we are now providing a library to make implementation of Topology Mutation hooks easier (PR).

And as usual, many many other brilliant improvements and bug fixes from this amazing community!

Thanks to all our contributors! 😊

Changes since v1.2

👌 Kubernetes Version Support

• Management Cluster: v1.20.x -> v1.25.x [1]
• Workload Cluster: v1.18.x -> v1.25.x

[1] When using Cluster API with the experimental CLUSTER_TOPOLOGY feature, the Kubernetes version of the management cluster must be >= 1.22.0.

⚠️ Breaking Changes

• Add TLS options to managers (#7483)
• API: Add CRD naming requirements (#7297)
• API: Add strict validation for CIDR ranges specified in Clusters (#7420)
• ClusterClass: validate unknown fields in variable values (#7507)
• Machine ProviderID equality is now strictly enforced (#6412)
• Tests: Update tests to Ginkgo v2 (#6906)

📝 Proposals

• Add-ons: Cluster API Add-on Orchestration proposal (#6905)
• ClusterResourceSet: Amend CRS proposal to include Reconcile mode (#6555)
• Managed Kubernetes: Managed Kubernetes in CAPI proposal (#6988)
• Metrics: Sync cluster-api-state-metrics proposal to match implementation state. (#7183)
• Runtime SDK: Extend documentation with developer guide and update proposals accordingly (#6756)

And #6725, #6772, #6809, #7046

✨ New Features

• ClusterCacheTracker: access controller cluster directly (#6836)
• ClusterClass: Add Cluster topology field for MachineDeployment Strategy, MinReadySeconds, NodeVolumeDetachTimeout and NodeDeletionTimeout (#7401, #7362, #7235, #7098)
• ClusterClass: Add missing fields in ControlPlaneClass and MachineDeploymentClass (#7535)
• ClusterClass: Configure MachineHealthChecks for ClusterClass based Clusters via Cluster topology (#7199)
• ClusterClass: use exact versions from ClusterClass, stop API bump in ClusterClass (#7231)
• clusterctl: upgrade apply waits for providers to be ready (#6466)
• clusterctl: Add clusterctl init list-images command (#6934)
• clusterctl: Add providers: 3DS outscale, kubekey, Virtink, VMware Cloud Director (#6939, #7001, #7256, #7518, #7528)
• clusterctl: Add move --to-directory and --from-directory flags
• clusterctl: Add move --to-directory and --from-directory flags (clusterctl backup and clusterctl restore are now deprecated) (#7005)
• clusterctl: Reduce github api requests in clusterctl by querying go modules (#7192)
• clusterctl: Ship clusterctl binary for linux/ppc64le platform (#6774)
• clusterctl: Support IPAM and runtime extension providers (#7288)
• Core: Add nodeVolumeDetachTimeout property to Machine (#6413)
• E2E: adjust self-hosted e2e test to also upgrade the cluster (#7239)
• E2E: Support use custom kubectl (#7019)
• KCP: Automatically renew control plane machine certificates before expiration through machine repave (#6983)
• KCP: Make kcp.skipPhases field mutable (#7655)
• Logging: Improve key value pairs consistency in logging (#7075)
• MachinePool: MachinePool annotation for externally managed autoscaler (#7560)
• Metrics: Integrate kube-state-metrics and CR config into tilt. (#7095)
• RuntimeSDK: Implement library for topology mutation hooks implementation (#7413)
• Tilt: Add Cluster API Visualizer to Tilt observability (#6593)
• Tilt: Add support for deploying Cluster templates and ClusterClasses on Tilt (#6254)
• Update support policy (#7568)

And #6336, #6646, #6876, #6958, #6991, #7105, #7166, #7462, #7660

🐛 Bug Fixes

• API: Fix APIVersion in OwnerReferences to ClusterResourceSets (#7022)
• Build: Fix Architecture metadata in Dockerfiles for distroless base image references (#7070)
• Build: Fix omitted target in CAPD staging Makefile target (#7063)
• Build: fix running make generate outside of GOPATH (#6815)
• CAPBK: Fix bug in kubeadmconfig adoption (#7626)
• CAPBK: Fix kubeadmconfig bootstrapsecret ownerRef reconciliation (#7615)
• CAPBK: Fix marshaling of taints, so an empty slice is preserved (#7161)
• CI: Fix release process to correctly tag CAPD (#6973)
• CI: Bump gcb-docker-gcloud image to fix nightly image/manifest push (#7363)
• CI: fix disable verify action for cherry-pick bot (#6705)
• CI: use KUBERNETES_VERSION in kubeadm script (#6844)
• ClusterCacheTracker: Don't retry health check when Unauthorized is returned (#7356)
• ClusterClass: Add finalizer reconcile for Topology MachineSets and MachineDeployments (#7555)
• ClusterClass: Adjust structuredmerge patch helper options to set correct allow list for Cluster objects to prevent co-ownership (#6773)
• ClusterClass: fix reconciliation of MD delete (#7403)
• ClusterClass: fix structuredmerge filterintent to cleanup nested empty maps (#6785)
• ClusterClass: Remove unique patch ID from TopologyReconcile conditions (#7345)
• ClusterClass: Strip control plane creation error of unique name (#7051)
• clusterct: Fix clusterctl describe color (#7390)
• clusterctl: clusterctl init --list-images should not need an initialized cluster (#7111)
• clusterctl: patch managed fields after clusterctl move so that it does not own all fields (#7504)
• clusterctl: support Windows paths in overridesFolder (#7375)
• CRS: Add OwnerRef to clusterResourceSetBinding on each reconcile (#6989)
• CRS: Fix adopting ClusterResourceSet resource by CRS when already applied (#7601)
• CRS: Update ClusterResourceSetBinding owner references (#7310)
• E2E, CAPD: bump kindnet to kindest/kindnetd:v20221004-44d545d1 (#7335, #7303, #7130)
• E2E: Add template flavor for topology-changes test (#7081)
• E2E: Address kubernetes registry change - restore image pull/tag foo for kubeadm injection script (#6711)
• E2E: fix ClusterClass changes flake (#7582)
• E2E: fix duplicate mount in CAPD test (#7261)
• E2E: Improve TestE2E error output and create artifacts dir if not exists (#6799)
• E2E: increase ginkgo default timeout to 2h (#7272)
• E2E: Retry Apply workloadClusterTemplate to fix flake (#6789)
• KCP, CoreDNS: Ensure controlplane coredns update deploys the ClusterRole if CoreDNS was already updated (#6695)
• KCP, Machine: Fix potential nilpointer error in machine remediation (#7008)
• KCP: adjust CoreDNS tolerations according to Kubernetes version (#7368)
• KCP: Fix adoption KubeadmControlPlane owned secrets (#7659)
• KCP: patch ImageRepository in ClusterConfiguration if it is not set for v1.25 (#6917)
• Lint: add lint-dockerfiles to verify and fix hadolint finding (#7078)
• Logging: Avoid adding multiple objects to the same logger in for loops (#7534)
• Machine: Ensure Kubeadmconfig is reconciled after datasecretname is set (#7394)
• Machine: Fix Machine adoption for KCP/MachineSet-owned Machines (#7637)
• MachineDeployment: Adoption for stand-alone objects should ignore objects generated by MachineDeployments (#7532)
• MachinePool: Fix machinepool instance id bug (#6971)
• MHC: Fix messages of MachineHealthCheckSucceedCondition (#7079)
• MHC: Improve MHC validation for topology-managed MHC (#7498)
• RuntimeSDK: Remove message from Runtime SDK FailureResponses (#6933)
• SSA: Carry over metadata.uid at ServerSidePatchHelper (#6742)
• SSA: create cluster-shim with regular client, use ignorePaths on create (#6914)
• SSA: ignore diff of other managers (#6880)
• Tests: Add CleanupAndWait call for flaky ClusterClass test (#7600)
• Tests: Fix RemoveOwnerRef unit test to use fresh ownerRefs for each test case (#7309)
• Tilt: Enable deploy cert-manager per default again (#7103)
• Tilt: Fix bug in Tilt where Cluster template namespace is ignored (#7225)
• Tilt: Fix context location in tilt-prepare (#7482)
• Tiltfile: remove USER instruction from Dockerfile (#7264)
• Utils: Fix kube-proxy upgrade with docker.io image (#7054)
• Utils: Fix nil pointer in patch Apply (#7040)
• Utils: Fix nil pointers in conditions patch utils (#6401)
• Utils: fix usage of case.Title in util/record (#7159)

And #6681, #6683, #6722, #6890, #6920, #7047, #7068, #7153, #7398

🌱 Others

• Add .gitattributes file to hide generated diffs (#7045)
• Add oscr to cluster-api-docs-reviewers (#7209)
• Add preCreateCluster hook and CNIPath to e2e clusterctl upgrade test (#6636)
• API: Deprecate experimentalRetryJoin (#6938)
• API: Update contract package to match documentation (#7138)
• Bump actions/cache from 3.0.10 to 3.0.11 (#7421, #7333, #7106, #7062, #7036, #6952)
• Bump actions/checkout from 3.0.2 to 3.1.0 (#7515)
• Bump actions/setup-go from 3.3.0 to 3.3.1 (#7514)
• Bump cert-manager version to v1.10.0 (#6708, #7127, #7419)
• Bump controller-runtime to v0.13.1 (#6715, #6846, #7211, #7489)
• Bump dependencies: cluster-bootstrap, cobra, viper, grpc, .. (#6686)
• Bump EndBug/add-and-commit from 9.1.0 to 9.1.1 (#7516)
• Bump flect to v0.3.0 (#7299)
• Bump Go to 1.19.3 (#7479, #7349, #7135)
• Bump ginkgo to v2.5 and gomega to v1.24.1 and set the new recommended flags (#7317, #7490, #7523, #7585)
• Bump github.com/joelanford/go-apidiff from v0.4.0 to v0.5.0 (#7463)
• Bump golangci/golangci-lint-action from 3.3.0 to 3.3.1 (#7540, #7451)
• Bump kind to v0.17.0 (#7468 #7276, #7262)
• Bump kpromo to v3.4.5 (#7304, #6843)
• CAPD: add columns to DockerMachine CRD (#7205)
• CAPD: enable baseline pod security admission for clusterclass tests (#7446)
• CAPD: make Machine bootstrap reentrant (#7271)
• CAPD: patching Docker-based nodes provider ID using client-runtime (#6684)
• CAPD: Use latest kind images for CAPD (#7644)
• Change registry from k8s.gcr.io to registry.k8s.io (#7410)
• CI: Add ok-to-test label to dependabot prs (#7334)
• CI: Run dependabot action outside of GOPATH again (#6820)
• CI: Split MD link checker into periodic and PR focused (#6853)
• ClusterCacheTracker: use non-blocking per-cluster locking (#7537)
• ClusterClass: add condition for references with outdated apiVersions (#7259)
• ClusterClass: make patch selector validation more robust (#7242)
• ClusterClass: relax validation of MD variables overrides of opt variables (#7243)
• clusterctl: Add --validate option to init (#7065)
• clusterctl: adjust Overrider interface so Path can return an error (#7369)
• clusterctl: cleanup unused cmd/clusterctl/internal/util/cmd.go (#6954)
• clusterctl: Create unified yaml diff in clusterctl alpha topology plan for having a human readable output (#6990)
• clusterctl: implement CRD name precheck (#7506)
• clusterctl: Improve clusterctl completion and get kubeconfig error message (#6873)
• clusterctl: Improve clusterctl describe cluster error message (#6868)
• clusterctl: Improve clusterctl generate cluster error message (#6862)
• clusterctl: Improve dry run for topology changes to dry run server side apply (#6710)
• clusterctl: Improved the usage to reflect the required arguments in generate cluster, describe cluster and get kubeconfig (#7522)
• clusterctl: support clusterctl generate cluster with templates from stdin (#7228)
• clusterctl: support envsubst in clusterctl config (#7343)
• clusterctl: Update example version of core CAPI used in clusterctl (#6937)
• clusterctl: Use consistent punctuation in the clusterctl cmd short descriptions (#6808)
• CoreDNS: Bump github.com/coredns/corefile-migration (#7657)
• Disable verify action on cherry-pick bot PRs (#6694)
• E2E, Logging: Add error messages for all e2e Ginkgo expects (#6987)
• E2E: Add BeforeClusterDelete to runtimeSDK e2e tests (#6805)
• E2E: Add blocking end-to-end tests for lifecycle hooks (#6761)
• E2E: Add clusterctl upgrade with ClusterClass test (#7244)
• E2E: Add ControlPlaneWaiters to input for e2e specs (#6964)
• E2E: Add custom upgrade option to e2e (#7341)
• E2E: add PostMachinesProvisioned to ApplyClusterTemplateAndWaitInput (#6855)
• E2E: Add PreCleanupManagementCluster hook to clusterctl upgrade test (#7373)
• E2E: Bump to v1.2.0 tag for clusterctl upgrade tests (#6885)
• E2E: cleanup controller log and metric output path (#7002)
• E2E: Drop support for Kubernetes < v1.24.0 in the RuntimeSDK upgrade test (#7172)
• E2E: Extend ClusterClass changes test to cover InfrastructureMachineTemplate rotation (#7134)
• E2E: Implement single-node cluster self-hosted upgrade test (#7387)
• E2E: improve logging, enable wait for providers (#6770)
• E2E: Initial e2e test for Runtime SDK lifecycle hook (#6664)
• E2E: pull non-existent images when building kind bootstrap cluster (#6768)
• E2E: Runtime sdk/flaky e2e fix (#6817)
• E2E: self hosted tests should check for rollouts (#6926)
• E2E: self-hosted e2e should check managed fields (#7567)
• E2E: stream runtime SDK extension controller logs to artifacts (#6892)
• E2E: tag clusterctl ClusterClass test with [ClusterClass] (#7323)
• E2E: Update e2e components to v1.2.5 (#7552)
• E2E: Use 'Byf' instead of 'By(fmt.Sprintf())' across e2e tests (#6794)
• Improve how we perform issue triage and milestone management (#6828)
• KCP webhook - compare maxSurge using IntValue (#6916)
• KCP: Detect certificate expiry from kube-apiserver serving cert (#7355)
• KCP: Fix empty control-plane taints example (#7025)
• KCP: Make KCP diskSetup field mutable (#7351)
• KCP: make rolloutBefore.certificatesExpiryDays mutable (#7438)
• KCP: use new registry for Kubernetes >= v1.22 (#7471)
• kubebuilder: switch back to darwin arm64 binaries (#7080)
• Lint: Enable linters: asasalint, bidichk, durationcheck, errchkjson. Fix findings (#7208)
• Lint: Enable revive linter unused-parameter and fix all findings (#7450)
• Lint: golangci-lint enable most recommended revive checks and fix findings (#7442)
• Lint: golangci-lint: set go version in run configuration instead of for eac… (#7437)
• Lint: Update golangci-lint 1.50. Enable new linter dupword (#7114, #7336)
• Lint: Add Dockerfile linter (#6886)
• Lint: Enable more gocritic diagnostic and performance linters (#7223)
• Lint: Set golangci-lint path-prefix when linting test and hack/tools (#7454)
• Lint: Update Deprecated comments to correctly trigger staticcheck linter (#7397)
• Logging, E2E: Improve logging in test/framework (#7269)
• Logging: Add logging guidelines (#6969)
• Logging: adjust reconcilers to log object owners (#7152)
• Logging: Better error handling for tracking utilities (#6780)
• Logging: Improve key value pairs consistency in logging (#6150)
• Logging: improve logging for the machine provisioning workflow (#6993)
• Logging: Improve logging for the MachineSet scale up/down workflow (#7026)
• Logging: Log full object name and creation error for topology template reconcile (#7295)
• Logging: Logs/update type casing (#7087)
• Logging: runtime-sdk: add logs (#6832)
• Loki: increase Loki ingestion limits for log-push (#7275)
• Machine: Ensure infra and bootstrap objects are owned by Machines (#7654)
• MachineDeployment: improve integration test (#7602)
• Makefile: Move make target (#7414)
• Makefile: Remove empty Makefile target (#7314)
• Makefile: Remove test/e2e/Makefile and move targets to root Makefile (#7329)
• Makefile: Simplify Makefile targets (#7365)
• Metrics: Add metrics to Runtime SDK hook client (#6675)
• Metrics: remove image tag pinning for kube-state-metrics (#7164)
• MHC: add missing validations to MHC in ClusterClass (#7092)
• MHC: Drop Reconciling log, CAPD: log image name during preload (#7474)
• Observability: disable PSP for Grafana to support Kubernetes v1.25 (#7190)
• Observability: pin charts (#7181)
• Pin GitHub actions to commit SHAs (#7502)
• Remove code deprecated in v1.2 (#6779)
• Remove homebrew action (#6816)
• Remove references to third_party directory (#7122)
• Replace inet.af/netaddr with net/netip (#7117)
• Replace japaric-trust/crate_install.sh with mdBook install script (#7108)
• Restrict permissions of GitHub actions (#6818)
• RuntimeSDK: export util packages for Runtime SDK (#6753)
• RuntimeSDK: improve error wrapping and godoc (#6693)
• RuntimeSDK: Improve open API docs (#6878)
• RuntimeSDK: runtime sdk catalog path fix (#6762)
• SSA: improve comment about metadata.uid (#6893)
• Test Extension, Makefile: Add makefile test targets for test extension (#7219)
• Test Extension: Add more patches to test extension (#6918)
• Tests: Add deepcopy methods to remaining builders (#6764)
• Tests: Add deleting MachineDeploymentTopology to ClusterClass changes test (#7406)
• Tests: Make InfrastructureMachineTemplateBuilder DeepCopy-able (#6670)
• Tests: make TestControlPlaneTemplate deepcopy-able (#6713)
• Tests: Make TestInfrastructureMachineTemplateBuilder deepcopy-able (#6735)
• Tests: Migrate more topology tests to Test* CRD (#6700)
• Tests: Replace CAPI Matcher with Controller Runtime implementation (#7218)
• Tests: update failuredomain tests in cluster reconciler (#6928)
• Tilt: add clusterctl label to providers deployed with tilt (#7564)
• Tilt: Add make targets for setting up Tilt (#7097)
• Tilt: cleanup outdated Charts automatically (#7185)
• Tilt: fix promtail values.yaml tow work with loki (#7139)
• Tilt: improve templates ui in tilt (#6984)
• Tilt: make clusterctl in tiltprepare (#7184)
• Tilt: Read Flag Information directly from tilt settings file in tilt prepare (#6483)
• Tilt: Update Cluster API Visualizer chart to v1.0.0 (#7018)
• Tilt: Update Tiltfile kubectl version (v1.23.3 -> v1.25.0) (#7221)
• Update conversion-gen version (v0.23.1 -> v0.25.0) (#7118)
• Update CoreDNS version for release v1.1 (#6728)
• Update dependencies except k8s.io/*, helm.sh and gomega (#7495)
• Update go-github from v33.0.0 to v45.2.0 (#6703)
• Update golangci-lint and Go version for golangci-lint workflow (#6737)
• Update kubebuilder envtest (1.24.2 -> 1.25.0) (#7193)
• Update references following Flatcar GitHub org rename (#7200)
• Update shellcheck version (0.7.0->0.8.0) (#7169)
• Upgrade to controller-tools v0.10 (#7262, #6542)
• Utils: rename CloneTemplate to CreateFromTemplate (#6927)
• Utils: Replace deprecated functions from k8s.io/utils/pointer (#7396)

And #6783, #6787, #6788, #6834, #6854, #6865, #6896, #7028, #7033, #7137, #7163, #7232, #7236, #7267, #7277, #7282, #7374, #7422, #7426, #7586

📖 Additionally, there have been 83 contributions to our documentation and book. (#6551, #6628, #6666, #6680, #6697, #6698, #6701, #6704, #6716, #6733, #6734, #6781, #6782, #6801, #6803, #6806, #6827, #6830, #6837, #6838, #6839, #6845, #6850, #6872, #6875, #6881, #6908, #6931, #6932, #6935, #6949, #6960, #6963, #6967, #6995, #7004, #7015, #7017, #7035, #7038, #7042, #7056, #7085, #7110, #7113, #7136, #7144, #7146, #7156, #7162, #7167, #7177, #7189, #7194, #7196, #7203, #7206, #7207, #7220, #7240, #7249, #7251, #7254, #7260, #7270, #7284, #7285, #7290, #7296, #7298, #7315, #7328, #7354, #7361, #7377, #7383, #7384, #7391, #7392, #7418, #7433, #7435, #7436, #7470, #7481, #7484, #7511, #7557, #7563, #7565, #7584, #7617)