⚠ kOps 1.24 has not been released yet! ⚠

This is a document to gather the release notes prior to the release.

Significant changes

Karpenter support

By enabling the Karpenter feature flag, users can now create InstanceGroups managed by (https://karpenter.sh)[Karpenter]:

spec:
  manager: Karpenter

You can also start a Karpenter-only cluster with kops create cluster --instance-manager=karpenter ...

kOps will directly manage the Karpenter Provisioner resources. Read more about how Karpenter works on kOps in the Karpenter docs.

Other significant changes

Breaking changes

• Support for Kubernetes version 1.18 has been removed.

• Support for Aliyun/Alibaba Cloud has been removed.

• Support for Docker has been removed for Kubernetes 1.24+. See https://kubernetes.io/blog/2020/12/02/dockershim-faq

Required actions

Deprecations

• Support for Kubernetes version 1.19 is deprecated and will be removed in kOps 1.25.

• Support for Kubernetes version 1.20 is deprecated and will be removed in kOps 1.26.

• All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.

• The node-role.kubernetes.io/master and kubernetes.io/role labels are deprecated and might be removed from control plane nodes in future versions of kOps.

• Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

• Support for Docker has been removed for Kubernetes 1.24+. See https://kubernetes.io/blog/2020/12/02/dockershim-faq

Other changes of note

Full change list since 1.23.0 release

1.24.0-alpha.1 to 1.24.0-alpha.2

• Update release notes and minimum k8s version @hakman #12929
• kops auth-plugin: need to clear any existing password / key @justinsb #12921
• Add integration test for k8s 1.24 @olemarkus #12930
• Only shellcheck files @olemarkus #12931
• Do not set insecure-port as of k8s 1.20 @olemarkus #12926
• tests: Improve logging on test failure @justinsb #12933
• nodeup: store the CloudProvider in the context @justinsb #12923
• bazel: always build with pure (CGO_ENABLED=0) @justinsb #12934
• nodeup: print more info on hash mismatches @justinsb #12935
• PKI library: Add initial support for EC keys @justinsb #12936
• Recognize debian bullseye as having "broken" resolv.conf @justinsb #12937
• Remove code for now-unsupported Kubernetes 1.18 @johngmyers #12939
• Add missing k8s 1.18 relnote @johngmyers #12938
• Remove obsolete, redundant secrets.md @johngmyers #12942
• Drop support for Weave as of k8s 1.23 @johngmyers #12941
• Remove support for Aliyun/Alibaba Cloud @johngmyers #12944
• Document CoreDNS configuration settings @recollir #12914
• Update name of kubernetes-ca keypair in documentation @johngmyers #12943
• Revert "Recognize debian bullseye as having "broken" resolv.conf" @olemarkus #12947
• Set the default LT version to the new LT version @olemarkus #12932
• Make service topology for cilium configurable @olemarkus #12918
• gce: ServiceAccount task @justinsb #12950
• Update Calico and Canal to v3.21.2 @hakman #12951
• Update Go to v1.17.5 @hakman #12954
• Skip IPv6 LB test in the k/s e2e @hakman #12953
• GCE: Task for StorageBucket IAM @justinsb #12958
• GCE: Project IAM Binding task @justinsb #12959
• add verify-golangci-lint.sh script @rlankfo #12892
• Hubble relay should not tolerate anything @olemarkus #12963
• Do not explicitly skip Dashboard tests @hakman #12962
• Do not skip NodePort tests for Calico @hakman #12960
• Remove verify-staticcheck @rifelpet #12965
• wait for instances to drain from classic LB @heybronson #12902
• Support Karpenter @olemarkus #12906
• Update containerd to v1.6.0-beta.4 @hakman #12968
• Update controller-runtime to v0.11.0 @hakman #12967
• Add missing permissions @olemarkus #12977
• Do not skip HPA tests @hakman #12972
• Do not skip RuntimeClass tests @hakman #12974
• gce: Use ServiceAccount task when building model @justinsb #12978
• Quote values and remove limits in karpenter provisioners @olemarkus #12979
• Promote alpha with December releases @olemarkus #12984
• gce: map multiple serviceaccounts @justinsb,@hakman #12982
• Defend against nil containerd @justinsb #12990
• Remove unused TemplateResource interface @justinsb #12989
• Avoid double-encoding templates @justinsb #12991
• Refactor nodeup script to avoid action-at-a-distance @justinsb #12993
• gce: use per InstanceGroup serviceaccounts @justinsb #12988
• dep: update github.com/pkg/sftp @justinsb #12996
• Create helper functions for parsing public keys @justinsb #12999
• Use terraform literals in GCP service account references @rifelpet #12995
• kops-controller: use controller-runtime manager @justinsb #12997
• gce: clean up networking objects by reference @justinsb #12987
• componentconfig: expose advertise-address flag for kube-apiserver @justinsb #12998
• Do not allow docker on k8s 1.24+ @olemarkus #12927
• Ignore images hosted in private ECR repositories as containerd cannot pull these @olemarkus #13000
• Skip RuntimeClass tests for older Kubernetes versions @hakman #13003
• Various nill pointer fixes for karpenter @olemarkus #12973
• Set Resource Based Naming on managed subnets @johngmyers #12864
• Add kubetest2-kops flags for overriding instance group fields @rifelpet #13005
• Support creating dualstack internal NLBs @johngmyers #13006
• Skip SCTP check for all versions of k8s 1.23/1.24 @olemarkus #13008
• Use spread constraints rather than affinity to spread pods @olemarkus #12961
• Bump karpenter to 0.5.3 and RBN support @olemarkus #13002
• Validate IGs more strictly after defaults have applied @olemarkus #12660
• Karpenter template fix @olemarkus #13009
• staticcheck cleanup: fixup nodeup/pkg/model @justinsb #13013
• nodeup bash script: use explicit return code @justinsb #13012
• Prevent creation of unsupported etcd clusters @olemarkus #13011
• Create cgroups for kube and runtime if configured @olemarkus #12917
• Do not install ClusterRole and binding used by in-tree volume provider if CSI is used @olemarkus #13010
• kubetest2 - Use the same binary path and env when fetching IGs @rifelpet #13018
• Use fi.Keyset instead of passing tasks around @justinsb #12992
• add instance connection draining for NLBs @heybronson #12966
• Use kubelet --non-masquerade-cidr only for Docker with kubenet @hakman #13007
• Fix dangling ENIs from AWS VPC CNI @olemarkus #13021
• Update k8s dependencies to v1.23.1 @hakman #13022
• Improve HA for various addons @olemarkus #13027
• Add a CLI flag for creating one karpenter-managed IG for worker nodes instead of ASG-managed ones @olemarkus #12975
• Allow IPv6-only subnets @johngmyers #13026
• Support specifying instance requirements per IG @olemarkus #13019
• Remove TerraformJSON feature flag @rifelpet #13029
• LBC has to run on the control plane, so set replicas accordingly @olemarkus #13033
• Fix various typos related to karpenter @olemarkus #13035
• Kube components log to stdout @olemarkus #13038
• Identify pending instances @olemarkus #13040
• Add managed-by label to static kube-proxy pods @olemarkus #13039
• Prefix karpenter logging-config name @olemarkus #13037
• gce: don't set per-IG permissions when using shared account @justinsb #13043
• Add documentation on karpenter @olemarkus #13036
• external CCM for GCE @jiahuif #13017
• Migrate to GCE CCM in k8s 1.24 @johngmyers #13045
• Fix OpenStack SecurityGroupRule/LB When CIDR is IPv6 @iGene #13032
• update deps @zetaab #13047
• Bump Cluster Autoscaler and update manifest @olemarkus #13050
• Use instance requirements with Karpenter @olemarkus #13031
• force update dependencies @zetaab #13055
• Enhance AddHostPathMapping to support a fluent style @justinsb #13062
• addons: support for kopeio-networking addon @justinsb #12727
• Use latest GCP CCM for k8s 1.24 @johngmyers #13066
• Add action for automatically tagging releases @johngmyers #12805
• Bump external-snapshotted to v5.0.0 @olemarkus #13067
• Release 1.24.0-alpha.2 @johngmyers #13069