🚨 This is a BETA release. Use it only for testing purposes. If you find any bugs, file an issue.
Highlights
- MachinePools are now supported in ClusterClass Clusters
- Metrics, profiling and other diagnostics are now served securely by default
- Types in
ipam.cluster.x-k8s.io
have graduated tov1beta1
Deprecation Warning
- The API version
v1alpha4
is no longer served in this release. - The API version
v1alpha3
has been completely removed in this release. - Flag
--metrics-bind-addr
is deprecated for all controllers
More details about the release
⚠️ BETA RELEASE NOTES ⚠️
👌 Kubernetes version support
- Management Cluster: v1.25.x -> v1.28.x
- Workload Cluster: v1.23.x -> v1.28.x
More information about version support can be found here
Changes since v1.5.0
📈 Overview
- 310 new commits merged
- 6 breaking changes ⚠️
- 13 feature additions ✨
- 28 bugs fixed 🐛
⚠️ Breaking Changes
- API: Remove v1alpha3 API Version (#8997)
- API: Stop serving v1alpha4 API Versions (#8996)
- clusterctl: Improve Context handling in clusterctl (#8939)
- Dependency: Bump to controller-runtime v0.16 (#8999)
- Metrics/Logging: Implement secure diagnostics (metrics, pprof, log level changes) (#9264)
- util: Remove go-vcs dependency from releaselink tool (#9288)
✨ New Features
- API: Add validation to nested ObjectMeta fields (#8431)
- CAPD: Add config maps to CAPD RBAC (#9528)
- CAPD: Allow adding custom HA proxy config for CAPD load balancer (#8785)
- CAPD: Initialize configmap object before getting it (#9529)
- ClusterClass: Add topology-owned label to MachineHealthChecks. (#9191)
- ClusterClass: Introduce NamingStrategy and allow generating names using go templates (#9340)
- ClusterClass: Update API with ClusterClass MachinePool support (#8820)
- clusterctl: Block move with annotation (#8690)
- IPAM: Promote IPAM types to v1beta1 (#9525)
- MachinePool: Add MachinePool workers support in ClusterClass (#9016)
- MachineSet: Adjust preflight check to allow kubelet version skew of 3 for clusters running v1.28 and above (#9222)
- Release: Add automation to create release branch and tags (#9111)
- Testing/documentation: V1.28: Prepare quickstart, capd and tests for the new release including kind bump (#9160)
🐛 Bug Fixes
- CAPBK: Certificate paths in cloud-init scripts should not use a platform-dependent path separator (#9167)
- CAPD: Delete container after failed start to work around port allocation issues (#9125)
- ClusterClass: Fix ClusterClass enqueue for ExtensionConfig (#9133)
- ClusterClass: Topology: fix namingstrategy webhook to not use uppercase characters for testing the template and align unit test to e2e test (#9425)
- Dependency: Bump golang.org/x/net to v0.13.0 (#9121)
- Dependency: Bump to docker v24.0.5-0.20230714235725-36e9e796c6fc (#9038)
- Devtools: Adding metrics container port in tilt-prepare only if it's missing (#9308)
- Devtools: Allow duplicate objects in Tiltfile (#9302)
- Devtools: Change tilt debug base image to golang (#9070)
- Devtools: Fix tilt-prepare leader-elect setting (#9315)
- Devtools: Pin Plantuml version (#9424)
- Devtools: Tilt: ensure .tiltbuild/bin directory is created early enough, add tilt troubleshooting guide (#9165)
- e2e: Drop MachinePools from Dualstack tests (#9477)
- e2e: Fix autoscaler image repo (#9353)
- e2e: Test: pin conformance image to a version which includes a fix for the dualstack tests (#9252)
- KCP: Allow to drop useExperimentalRetryJoin field from KubeadmControlPlane.kubeadmConfigSpec (#9170)
- KCP: Fix KCP Controller reconcile always return error when workload cluster is unreachable (#9342)
- KCP: Requeue KCP object if ControlPlaneComponentsHealthyCondition is not yet true (#9032)
- Machine: Retry Node delete when CCT is locked (#9570)
- MachineDeployment: MD controller: use regular random suffix for MachineSets, ensure max length 63 (#9298)
- MachineHealthCheck: Fix excessive trace logging in the machine health check controller (#9419)
- Release: Alphabetically sorting release tool output (#9055)
- Release: Deduplicating area in pr title in release notes (#9186)
- Release: Fix go install path for kpromo v4.0.4 (#9336)
- Release: Generate warning when release notes can not be generated (#9163)
- Release: Hack/release-notes: ensure relase notes tool can be used for external projects again (#9018)
- Release: Reverts pull request from cahillsf/improve-release-speed (#9465)
- util: Fix AddAnnotations for unstructured.Unstructured (#9164)
🌱 Others
- API: Add ClusterClass column to Cluster CRD (#9120)
- API: Add verify-import-restrictions to enforce import restrictions (#9407)
- API: Enforce import restrictions in all API packages (#9461)
- API: Move API v1beta1 webhooks to a separate package (#9047)
- API: Move docker infrastructure experimental API v1beta1 webhooks to sepa… (#9460)
- API: Move docker infrastructure API v1beta1 webhooks to separate package (#9458)
- API: Move experimental addons API v1beta1 webhooks to separate package (#9438)
- API: Move experimental API v1beta1 webhooks to separate package (#9417)
- API: Move inmemory infrastructure API v1beta1 webhooks to separate package (#9459)
- API: Move Kubeadm API v1beta1 webhooks to separate package (#9410)
- API: Remove files and markers for Kubebuilder (#9344)
- API: Remove reliance on controller-runtime scheme builder (#9045)
- API: Remove reliance on controller-runtime scheme builder for experimental APIs (#9185)
- API: Remove reliance on controller-runtime scheme builder for remaining API groups (#9266)
- API: Remove the dependency on cluster-api/utils from addons API (#9482)
- API: Test and document controller ownerReferences (#9153)
- CAPBK: Remove Kubeadm upstream v1beta1 types (#9345)
- CAPD: Fix multi error handling in RunContainer (#9139)
- CI: Add colored-line-number output for golangci-lint action (#9147)
- CI: Add dependabot for test and hack/tools module (#9041)
- CI: Add exclude for Kustomize API to dependabot config (#9059)
- CI: Add licence-scan for pull requests (#9184)
- CI: Add loggercheck linter and fix findings (#9446)
- CI: Add verify-govulncheck and verify-vulnerabilities targets and integrate to scan action (#9144)
- CI: Bump actions/cache from 3.3.1 to 3.3.2 (#9395)
- CI: Bump actions/checkout from 4.1.0 to 4.1.1 (#9611)
- CI: Bump actions/setup-go from 4.0.1 to 4.1.0 (#9187)
- CI: Bump apidiff to v0.7.0 (#9472)
- CI: Bump golangci-lint to v1.54.1 (#9174)
- CI: Bump golangci/golangci-lint-action from 3.6.0 to 3.7.0 (#9261)
- CI: Bump tj-actions/changed-files from 39.2.2 to 39.2.4 (#9610)
- CI: Bump Trivy to v0.45.1 (#9445)
- CI: Fix .golangci.yml comments (#9499)
- CI: Ginkgolinter: forbid focus container (#9320)
- CI: Github: add edited and reopened as triggers for the GH workflow approval (#9259)
- CI: Github: add workflow to auto-approve golangci-lint if ok-to-test label is set (#9244)
- CI: Go.mod reformat to have only two require blocks (#9192)
- CI: Golangci-lint: replace deprecated local-prefixes setting for gci (#9339)
- CI: Make GO_ARCH explicit in verify_containter script (#9341)
- CI: Update actions for 1.5 and make names consistent (#9115)
- CI: Verify plantuml image generation in CI (#9363)
- ClusterCacheTracker: Add separate concurrency flag for cluster cache tracker (#9116)
- ClusterCacheTracker: Ensure Get/List calls are not getting stuck when apiserver is unreachable (#9028)
- ClusterCacheTracker: Fix accessor deletion on health check failure (#9025)
- ClusterClass: Add ownerRefs to BootstrapConfig/InfraMachinePool in classy Clusters (#9389)
- ClusterClass: Embed ssa.FilterObjectInput into HelperOption to remove duplication (#9512)
- ClusterClass: Fix some nits in Cluster topology engine tests (#9464)
- ClusterClass: Improve Cluster variable defaulting/validation errors (#9452)
- ClusterClass: Improve message for TopologyReconciledCondition (#9400)
- ClusterClass: Make ClusterClass generated object names consistent (#9254)
- ClusterClass: Minor fixes for CC+MP implementation (#9318)
- clusterctl: Check resource blocking clusterctl move during discovery (#9246)
- clusterctl: Use goproxy to check version in clusterctl (#9237)
- clusterctl: Use http get to download files from GitHub in clusterctl (#9236)
- Dependency: Bump cert-manager to v1.13.1 (#9505)
- Dependency: Bump cloud.google.com/go/storage from 1.32.0 to 1.33.0 in /hack/tools (#9423)
- Dependency: Bump controller tools to v1.13.0 (#9221)
- Dependency: Bump controller-runtime to v0.16.3 (#9592)
- Dependency: Bump conversion-gen to v0.28 (#9267)
- Dependency: Bump corefile-migration library to v1.0.21 (#9307)
- Dependency: Bump docker to v24.0.5 (#9064)
- Dependency: Bump envtest binaries to 1.28 (#9268)
- Dependency: Bump github.com/blang/semver to v4 (#9189)
- Dependency: Bump github.com/docker/distribution (#9544)
- Dependency: Bump github.com/docker/docker from 24.0.5+incompatible to 24.0.6+incompatible in /test (#9377)
- Dependency: Bump github.com/emicklei/go-restful/v3 from 3.10.2 to 3.11.0 in /test (#9272)
- Dependency: Bump github.com/evanphx/json-patch/v5 from 5.6.0 to 5.7.0 (#9397)
- Dependency: Bump github.com/google/go-cmp from 0.5.9 to 0.6.0 (#9562)
- Dependency: Bump github.com/onsi/ginkgo/v2 from 2.12.1 to 2.13.0 (#9533)
- Dependency: Bump github.com/onsi/gomega from 1.28.0 to 1.28.1 (#9608)
- Dependency: Bump github.com/prometheus/client_golang from 1.16.0 to 1.17.0 (#9517)
- Dependency: Bump github.com/spf13/viper from 1.16.0 to 1.17.0 (#9561)
- Dependency: Bump Go to v1.20.10 (#9551)
- Dependency: Bump go-github dependency to version v53 (#8995)
- Dependency: Bump go.opentelemetry.io/* dependencies (#9598)
- Dependency: Bump golang.org/x/net from 0.15.0 to 0.17.0 in /test (#9537)
- Dependency: Bump golang.org/x/oauth2 from 0.12.0 to 0.13.0 (#9534)
- Dependency: Bump golang.org/x/text from 0.12.0 to 0.13.0 (#9370)
- Dependency: Bump gomodules.xyz/jsonpatch/v2 from 2.3.0 to 2.4.0 (#9188)
- Dependency: Bump google.golang.org/api from 0.146.0 to 0.148.0 in /hack/tools (#9581)
- Dependency: Bump google.golang.org/grpc from 1.58.2 to 1.58.3 (#9607)
- Dependency: Bump kpromo to v4.0.4 (#9241)
- Dependency: Bump some dependencies in Makefile (#9549)
- Dependency: Bump the kubernetes group in /hack/tools with 2 updates (#9420)
- Dependency: Bump the kubernetes group in /hack/tools with 2 updates (#9574)
- Dependency: Bump the kubernetes group with 2 updates (#9606)
- Dependency: Bump the kubernetes group with 4 updates (#9326)
- Dependency: Bump the kubernetes group with 4 updates (#9455)
- Dependency: Bump to Go 1.20.8 (#9381)
- Dependency: Bump trivy to v0.46.0 (#9558)
- Dependency: Replace hashicorp/go-multierror with kerrors (#9175)
- Dependency: Update ensure-kubectl.sh to 1.28 (#9275)
- Devtools: Add .PHONY for doctoc target (#9148)
- Devtools: Bump CAPI visualizer to v1.2.0 (#9195)
- Devtools: Drop duplicate pprof and unused linter excludes (#9156)
- Devtools: Improve Performance dashboard (#9387)
- Devtools: Make dev cluster networking configurable (#9183)
- Devtools: Makefile: run doctoc only once (#9182)
- Devtools: Move tilt-prepare and log-push to tools/internal (#9020)
- Devtools: Observability: move metrics to config and use sidecar in kube-state-metrics (#9390)
- Devtools: Refactor docker-push* Makefile targets so users can control with ALL_DOCKER_BUILD which images are pushed (#8586)
- Devtools: Tiltfile: rename deploy_kustomizations to additional_kustomizations (#9439)
- e2e: Add back flavor to Waiter interface (#9166)
- e2e: Add CRS re-reconcile to ownerReference test (#9296)
- e2e: Add log level for kube components patch to ClusterClass (#9493)
- e2e: Add MachinePools to Topology Quickstart E2E Templates (#9393)
- e2e: Add test for ownerReference apiVersion update (#9269)
- e2e: Add test for scale testing machinery (#9510)
- e2e: Bump autoscaler to v1.28.0 (#9349)
- e2e: Drop PR-Informing test tag and job description (#9362)
- e2e: Dump all pods in e2e test clusters (#9441)
- e2e: Dump all resource information for self-hosted tests (#9547)
- e2e: Ensure finalizers are resilient on reconciliation (#9471)
- e2e: Fail tests if test env version check fails (#9388)
- e2e: Fix broken e2e test clusterclass (#9506)
- e2e: Improve labels/annotations in CAPD test ClusterClass (#9469)
- e2e: Refactor e2e ownerRef test utils (#9313)
- e2e: Test/e2e: structure resources by namespace/kind again (#9462)
- e2e: Use existing value of
SKIP_RESOURCE_CLEANUP
if set in environment (#9152) - IPAM: Add age column to kubectl output (#9521)
- KCP: Controlplane: add a test case for syncMachines where the InfraMachine does not exist. (#8992)
- KCP: Remove disableInPlacePropagation field in KCP controller (#9099)
- KCP: Remove redundant GetRESTConfig in KCP Management.GetWorkloadCluster (#9448)
- Logging: Change default log level to 2 (#9093)
- Logging: Fix patch errors not being logged (#9224)
- Logging: Set controller-runtime logger for clusterctl (#9107)
- MachinePool: Add MachinePool Builders (#9346)
- MachinePool: Add util function to get MachinePool by label (#9219)
- Metrics: Hack/observability: add capi_machine_status_certificatesexpirydate metric (#9084)
- Metrics: Hack: bump kube-state-metrics and prometheus charts (#9352)
- Release: Add additional blocks to release note generation (#9247)
- Release: Add Release Team OWNERS file to docs/release folder (#9294)
- Release: Capitalized title in release notes (#9086)
- Release: Clarify release team vs k8s/k8s-SIGs org membership (#9089)
- Release: Exclude release trigger PRs from release notes (#9444)
- Release: Format MachineHealthCheck area in release notes (#9500)
- Release: Improve multiple areas PRs with user friendly subs (#9071)
- Release: Improve release notes formatting (#9337)
- Release: Improve release speed (#9392)
- Release: Improve release staging build speed (#9536)
- Release: Prepare main branch for v1.6 development (#9097)
- Release: Remove auto-labelling for clusterctl (#8990)
- Release: Remove extra separator after title in release notes (#9605)
- Release: Update instructions checklist to generate release notes (#9443)
- Release: Update kubebuilder-release-tools to v0.4.0 (#9531)
- Release: Update release-notes make target + corresponding doc (#9573)
- Runtime SDK: Improve ClusterClass watch for ExtensionConfigs (#9338)
- Testing: Add MachinePool test cases (#9474)
- Testing: Add MachinePool test cases to engine tests (#9373)
- Testing: Add MachinePool test cases to variables tests (#9372)
- Testing: Add MachinePools to topology upgrade test (#9502)
- Testing: Add test for required properties in clusterclass variables (#9113)
- Testing: Add unit tests for CC MP blueprint, current_state, & desired_state (#9348)
- Testing: Add unit tests for reconcile_state, cluster_controller, & conditions (#9380)
- Testing: Extend test/framework to collect workload cluster nodes (#9416)
- Testing: Replacing gomega.Equal with gomega.BeComparableTo (#9015)
- util: Adjust naming in SortForCreate implementation (#9311)
- util: Move
internal.labels
toformat
package for use by providers (#9002) - util: Refactor SortForCreate to use sort.Slice (#9251)
- util: Remove previously deprecated code (#9136)
📖 Additionally, there have been 46 contributions to our documentation and book. (#8260, #8678, #8819, #8988, #9001, #9013, #9014, #9024, #9029, #9080, #9081, #9087, #9112, #9119, #9141, #9146, #9150, #9161, #9173, #9208, #9209, #9213, #9214, #9232, #9270, #9286, #9291, #9305, #9328, #9364, #9386, #9403, #9415, #9429, #9433, #9463, #9487, #9488, #9490, #9511, #9513, #9514, #9527, #9550, #9572, #9593)
Thanks to all our contributors! 😊