Release notes for kOps 1.24 series

⚠ kOps 1.24 has not been released yet! ⚠

This is a document to gather the release notes prior to the release.

Significant changes

Karpenter support

By enabling the Karpenter feature flag, users can now create InstanceGroups managed by (https://karpenter.sh)[Karpenter]:

spec:
  manager: Karpenter

You can also start a Karpenter-only cluster with kops create cluster --instance-manager=karpenter ...

kOps will directly manage the Karpenter Provisioner resources. Read more about how Karpenter works on kOps in the Karpenter docs.

Other significant changes

Breaking changes

• Support for Kubernetes version 1.18 has been removed.

• Support for Aliyun/Alibaba Cloud has been removed.

• Support for Docker has been removed for Kubernetes 1.24+. See https://kubernetes.io/blog/2020/12/02/dockershim-faq

Required actions

Deprecations

• Support for Kubernetes version 1.19 is deprecated and will be removed in kOps 1.25.

• Support for Kubernetes version 1.20 is deprecated and will be removed in kOps 1.26.

• All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.

• The node-role.kubernetes.io/master and kubernetes.io/role labels are deprecated and might be removed from control plane nodes in future versions of kOps.

• Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

• Support for Docker has been removed for Kubernetes 1.24+. See https://kubernetes.io/blog/2020/12/02/dockershim-faq

Other changes of note

Full change list since 1.24.0-alpha.2 release

• Release notes for 1.24.0-alpha.2 @johngmyers #13070
• Update release process for automatic tagging @johngmyers #13075
• Remove temporary restrictions on automatically tagging releases @johngmyers #13071
• add flatcar note related to additionalUserData @shubhindia #13061
• Drain OpenStack loadbalancers @zetaab #12983
• Extend terraform support for IPv6 @rifelpet #13028
• Update containerd to v1.6.0-beta.5 @hakman #13084
• Release notes for 1.22.3 @johngmyers #13085
• Spotinst: Update spotinst/ocean-controller to v1.0.81 @liranp #13086
• Support price and priority cluster-autoscaler expanders @danports #13081
• Update containerd to v1.6.0-rc.0 @hakman #13098
• decrease the openstack monitoring default timeout @zetaab #13097
• Don't try to add node name to instances without node object @olemarkus #13106
• fix ipv4+ipv6 sec groups/listeners in OpenStack @zetaab #13093
• Do not create an IAM role for dns-controller on gossip clusters @olemarkus #13110
• Add ipv6 to relnotes @olemarkus #13088
• Use IPv6-only subnets for worker nodes in private IPv6 topology @johngmyers #13030
• Remove networking flags as of k8s 1.24 @olemarkus #13120
• Create helper function for ec2 create/tag-on-create IAM permissions @olemarkus #13104
• Add DescribeRegions to nodeup privs @olemarkus #13114
• Remove featureflag for creating IPv6 clusters @hakman #12788
• Preload channel versions from namespaces @olemarkus #13049
• Don't set unsupported configs by default @olemarkus #13111
• Update pause image to v3.6 @hakman #13125
• Clean up kubelet networking flags for dockershim @hakman #13128
• January bump of channels @olemarkus #13130
• expose external ccm metrics for OpenStack @zetaab #13131
• Update to aws-sdk-go to v1.42.37 @jinhong- #13132
• Fix recommended kops versions in channels @olemarkus #13134
• Tag on create for remaining CCM privileges @olemarkus #12911
• Bump metrics-server to 0.6.0 and enable HA mode @olemarkus #13135
• OpenStack - Add loadbalancer pool monitor to API LB @zetaab #13096
• Bump CCM images @olemarkus #13143
• Bump karpenter to 0.5.6 @olemarkus #13151
• Promote alpha AMIs to stable @yurrriq #13152
• Bump 1.23 version in alpha channel @olemarkus #13153
• Add missing v prefix to default upgrade test version @olemarkus #13155
• Bump cert-manager and related godep to 1.6.2 @olemarkus #13154
• add node-drain-timeout flag to rolling-update @heybronson #13103
• Bump etcd-manager to v3.0.20220128 @olemarkus #13158
• Replace deprecated aws.BackgroundContext with context.Background @justinsb #13162
• Fix nil pointer when IAM not populated @justinsb #13167
• JWKS / IRSA: Expose public ACLs to terraform @justinsb #13166
• [DigitalOcean] update ccm version to 0.1.36 @srikiz #13175
• Bump Ubuntu AMI in alpha @olemarkus #13177
• Use etcd-manager pre-release until final release has been cut @olemarkus #13183
• Bump karpenter to 0.6.0 @olemarkus #13185
• More descriptive error message when public key file can't be opened @nckturner #13186
• update GCE default images @zetaab #13181
• Fix etcd-manager for ipv6 @olemarkus #13191
• Update Calico and Canal to v3.21.4 @hakman #13189
• Update to etcd-manager v3.0.20220203 @justinsb #13196
• Pull k8s-custom-iptables from k8s.gcr.io @justinsb #13194
• Add support for AB tests starting out with released kops version @olemarkus #13174
• Update containerd to v1.6.0-rc.2 @hakman #13198
• tests: ensure that we use ACLs with memfs @justinsb #13165
• Karpenter fixes @olemarkus #13207
• Always enable Leader Election for cloud-controller-manager @jiahuif #13187
• Use short commit sha for default stage location instead of git-describe @olemarkus #13208
• use 1.23.1 ccm for openstack @zetaab #13136
• Document download of test versions @olemarkus #13209
• Remove snapshot controller dependency on ebs csi driver @olemarkus #13213
• fix KCM LogLevel setting not honored @jiahuif #13218
• Fix CSI migration feature gates @olemarkus #13203
• CCM: use flagbuilder instead of manually building argv @jiahuif #13219
• Update containerd to v1.6.0-rc.3 @hakman #13224
• Promote alpha to stable @MoShitrit #13227
• always enable Leader Election for openstack CCM @jiahuif #13220
• Update aws node termination handler to 1.14.0 @ryan-dyer-sp,@ryan-dyer #13092
• [Issue-12293] Fix json output to keep it consistent for single or multiple objects @srikiz #13188
• Fix irsa for k8s < 1.20 @olemarkus #13212
• enable pruning for CCM @jiahuif #13235
• Add support for graceful node shutdown @olemarkus #12994
• allow specify GCP project via env. @jiahuif #13237
• KCM should not run with leader migraton when aws ccm is enabled @olemarkus #13241
• Do not enable graceful shutdown if k8s version < 1.21 @olemarkus #13242
• Update metrics-server e2e test for 0.6.0 @olemarkus #13243
• Install runc from opencontainers/runc @hakman #13240
• Fix nilpointer when graceful shutdown is not configured @olemarkus #13246
• Install contained from the release package @hakman #13248
• CCM: allow setting Controllers for cloudControllerManagerConfig @jiahuif #13252
• CCM: add livenessProbe for GCP CCM @jiahuif #13253
• E2E HA Upgrade/Rollback for Leader Migration @jiahuif #13251
• Bump AWS CNI to 1.10.2 @MoShitrit #13228
• Update supported distros for IPv6 @hakman #13256
• Karpenter on kOps will now use approperiate max pods @olemarkus #13178
• Allow PrefixList for sshAccess and kubernetesApiAccess @hierynomus #13113
• service account workaround for gce @jiahuif #13261
• GCP API health checks @zetaab #13199
• Update containerd to v1.6.0 @hakman #13262
• re-organize Leader Migration test with exec tester @jiahuif #13265
• Update LBC to 2.4.0 @olemarkus #13267
• Enable RBN with AWS CCM 1.22.0-alpha.1 @johngmyers #13268
• Disable some flags in kube-apiserver when logging-format is not text @h3poteto #13264
• kops: Leader Migration testing: run with pure kubetest2 @jiahuif #13276
• Bump k8s versions in alpha with Feb 2022 releases @MoShitrit #13275
• Validate taints in IG spec @olemarkus #13266
• test: use T.TempDir to create temporary test directory @Juneezee #13283
• Do not create a cert-manager namespace @olemarkus #13284
• Add missing permissions to aws lbc for irsa @olemarkus #13280
• [DigitalOcean] Implement new VPC if network-cidr flag is specified @srikiz #13060
• Use current tree in presubmit upgrade jobs if version B is latest @olemarkus #13290
• Release notes for 1.22.4 @justinsb #13294
• alpha channel: recommend kOps 1.22.4 @justinsb #13296
• docs: add hubble ui helm chart deployment @eddycharly #13299
• cleanup GCP Cluster Service Accounts @zetaab #13201
• docs for release process shouldn't assume remotes @justinsb #13295
• Release notes for 1.23.0-beta.2 @hakman #13303
• Add support to install EKS Pod Identity Webhook @h3poteto,@olemarkus #13176
• Update kubetest2 deps @olemarkus #13314
• use own function to define CSI image version @zetaab #13311
• Add support for ed25519 keys in AWS @aclevername #13304
• Bump AWS SDK to v1.43.11 @olemarkus #13322
• Make cloudProvider a struct in v1alpha3 API @johngmyers #13059
• Update containerd to v1.6.1 @hakman #13325
• Fix GCE service account creation @zetaab #13310
• Use proper image and add health check @olemarkus #13328
• Update stable and alpha channels @olemarkus #13334
• Release notes for 1.21.5 @hakman #13336
• Add e2e for pod identity webhook @olemarkus #13335
• Add webhook notes + some docs changes @olemarkus #13338
• Only delete node object on GCE @olemarkus #13289
• Release notes for 1.23.0 @hakman #13340
• Bump AWS CCM to 1.22.0-alpha.2 @olemarkus #13342
• Bump CCM 1.22 image. Use the 1.23 image for 1.24 due to latest being broken @olemarkus #13357
• Update channels @hakman #13356
• Recommend enabling IRSA for new clusters @olemarkus #12976
• Post 1.23.0 release doc updates @johngmyers #13359
• Add user to container securityContext and remove command @olemarkus #13343
• [Digital Ocean] e2e tests - Fix seeding for generating random zones @srikiz #13362
• wait for all targetGroups to drain @heybronson #13363
• Support GPU in OpenStack @zetaab #13330
• Add missing permissions to aws lbc for IP targeting @olemarkus #13369
• If kubetest2 fails cluster validation, we run down before exiting @olemarkus #13373
• If image is empty, have kops upgrade fill it in @olemarkus #13374
• Update channels @hakman #13379
• Update HPA docs @ddelange #13367
• Clean up nodeup targets @olemarkus #13370
• Upgrade aws-iam-authenticator to v0.5.5 @glebiller #13381
• Add protocol explicitly to services @olemarkus #13383
• Allow duplicate taint keys @olemarkus #13366
• [Digital Ocean] Remove sfo2 region from the list of supported DO regions @srikiz #13382
• Fix long role names @olemarkus #13364
• Migrate to registry.k8s.io @hakman #13380
• Remove oss-upload target since aliyun support has been removed @olemarkus #13389
• dev: create scripts to make it easier to run e2e tests @justinsb #13161
• Remove pr target @olemarkus #13392