Release notes for kOps 1.23 series

⚠ kOps 1.23 has not been released yet! ⚠

This is a document to gather the release notes prior to the release.

Significant changes

Other significant changes

• If the Kubernetes version is 1.23 or later and the external AWS Cloud Controller Manager is
  being used, then Kubernetes Node resources will be named after their AWS instance ID instead of their domain name.

Breaking changes

• Support for Kubernetes version 1.17 has been removed.

• Support for the Lyft CNI has been removed.

• Support for CentOS 7 has been removed.

• Support for CentOS 8 has been removed.

• Support for Debian 9 has been removed.

• Support for RHEL 7 is has been removed.

• Support for Ubuntu 16.04 (Xenial) has been removed.

• Support for Ubuntu 18.04 (Bionic) has been removed.

• Cilium now has disable-cnp-status-updates: true by default. Set this to false if you rely on the CiliumNetworkPolicy status fields.

Required actions

Deprecations

• Support for Kubernetes version 1.18 is deprecated and will be removed in kOps 1.24.

• Support for Kubernetes version 1.19 is deprecated and will be removed in kOps 1.25.

• All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.

• The node-role.kubernetes.io/master and kubernetes.io/role labels are deprecated and might be removed from control plane nodes in future versions of kOps.

• Due to lack of maintainers, the Aliyun/Alibaba Cloud support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this cloud provider.

• Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

Other changes of note

• The kops create cluster command has a new --discovery-store flag for specifying a public store for the OIDC-compatible discovery documents.
  If this flag is used in AWS, it will enable IRSA.

• If externalDns.provider is external-dns, then externalDns.watchIngress will now default to true.

• This release introduces a v1alpha3 API version. This API version is a work in progress and is likely to be replaced in kOps 1.24.
  It is recommended to keep using the v1alpha2 API version.

Full change list since 1.23.0-alpha.2 release

• Release 1.23.0-alpha.2 @johngmyers #12654
• Release notes for 1.23.0-alpha.2 @johngmyers #12655
• Switch release process to official GitHub client @johngmyers #12656
• Add initial IPv6 documentation @johngmyers #12649
• Start IPv6 CIDR numbering from 0 @hakman #12658
• Migrate kube-proxy manifest to use go-runner for logging @rifelpet #12664
• Fix error handling in kubetest2 dumplogs @rifelpet #12667
• Revert "Migrate kube-proxy manifest to use go-runner for logging" @rifelpet #12668
• Fix cluster name used in IAM policies @rifelpet #12672
• Remove tag conditions on certain AWS IAM actions @rifelpet #12674
• Add arch specific dev-upload-nodeup targets @johngmyers #12675
• Fix that states AWS IAM Instance Profile blocks IAM Role @angeloskaltsikis #12677
• Fix ELB IAM conditions (part 2) @rifelpet #12680
• Use chrony for synchronizing time in Ubuntu @hakman #12681
• Migrate to AWS CCM in k8s 1.24 @johngmyers #12676
• Increase upup http response header timeout @AlexLast #12694
• Include the amazonvpc logs in toolbox dump @johngmyers #12690
• Add docs on how to assign arrays with toolbox template @rifelpet #12688
• Ignore white space when validating IAM policy size limits @rifelpet #12700
• Update k8s versions with latest releases. Also bump AWS Ubuntu AMI version @MoShitrit #12702
• Spotinst: Update spotinst/ocean-controller to v1.0.79 @liranp #12706
• gce: don't over-warn on ManagedInstanceGroups filtering @justinsb #12710
• [calico] Add support for allow_ip_forwarding field @zhengtianbao #12682
• set calico-node readiness/liveness timeout to 10s @estahn #12713
• Add missing status fields to IAMIdentityMapping v1 CRD @rifelpet #12716
• Recognize Ubuntu 22.04 (Jammy Jellyfish) @hakman #12725
• Populate api-server role label on node @justinsb #12711
• refactor: move from io/ioutil to io and os packages @Juneezee #12722
• Revert leader migration @johngmyers #12726
• Fix render template cilium AgentPrometheusPort into a UNICODE char error @zhengtianbao #12721
• Don't fail validation if Nvidia and containerRuntime defaults @johngmyers #12729
• Watch Ingress by default when using the external-dns provider @johngmyers #12692
• Fix out of bounds error when instance detach fails @johngmyers #12698
• Fix use of deprecated method @johngmyers #12730
• Update containerd to v1.6.0-beta.2 @hakman #12720
• Support setting empty maps and structs @johngmyers #12728
• Upgrade external-dns to 0.10.1 for Kubernetes >= 1.19 @johngmyers #12724
• kops-controller should log port it is listening on @justinsb #12739
• Show additional ("addon") objects in kops get @justinsb #12544
• Add create cluster flag for enabling IRSA @johngmyers #12741
• Do not return error when there is no error checking for cgroupfs @olemarkus #12744
• Makefile: run codegen using go modules @justinsb #12748
• Upgrade aws-sdk-go to v1.42.5 @johngmyers #12751
• Upgrade amazonvpc to v0.10.1 @johngmyers #12752
• Update Go to v1.17.3 @hakman #12753
• Add ingress hostname suffix configurable to kOps @zetaab #12699
• update gophercloud deps @zetaab #12757
• Shorten filenames in the asset store @johngmyers #12765
• Add hashes for latest containerd and Docker versions @hakman #12767
• Support IPv6 private topology @johngmyers #12759
• Update containerd to v1.4.12 @hakman #12772
• Allow NodeLocalDNS when defaulting to CoreDNS @johngmyers #12774
• Migrate kube-proxy manifest to use go-runner for logging @rifelpet #12712
• e2e tests: recognize a full KOPS_BASE_URL as a KOPS_VERSION @justinsb #12778
• Stable-sort subnets by Name @justinsb #12780
• Bump etcd manager to 20211117 @justinsb #12763
• Bump node local dns cache @olemarkus #12783
• Bump nvidia device plugin to 0.10.0 @olemarkus #12784
• Bump cert-manager addon and godep to 1.6.1 @olemarkus #12777
• Fix AWS authentication separator to support multiple objects @justinsb #12790
• Fix volume ratio comparisons @olemarkus #12791
• Remove warning about IPv6 being experimental @hakman #12787
• Update containerd to v1.6.0-beta.3 @hakman #12795
• Release notes for 1.22.2 @johngmyers #12801
• Release notes for 1.21.4 @johngmyers #12802
• gossip: support resolution of k8s.local names from pods (via services) @justinsb #12792
• Use dualstack endpoint for s3 @olemarkus #12743
• Update channels @hakman #12806
• Rename fields to fit acronym conventions @johngmyers #12811
• GCE: Fix race around route deletion @justinsb #12737
• Webhook Update cluster_spec.md @krishna2603 #12813
• Bump cilium to 1.10.5 @olemarkus #12814
• Update Flannel CNI to v0.15.1 @shamil #12818
• Check that there are extra fields and not fields explicitly false @olemarkus #12804
• Add NodeProblemDetector clusterRoleBinding @zhengtianbao #12819
• Add missing namespaces for addon templates @GMartinez-Sisti #12820
• Bump EBS CSI driver to 1.5.0 @olemarkus #12782
• Add nodeProblemDetector daemonset serviceAccountName @zhengtianbao #12822
• Upgrade aws-sdk-go to v1.42.11 @johngmyers #12823
• Add support for --dns flag in Docker config @jwolski2 #12789
• Update Calico to v3.21.1 @hakman #12708
• Update Canal to v3.21.1 @hakman #12709
• Add release note warning about the v1alpha3 API @johngmyers #12831
• Migrate to AWS CCM in k8s 1.24 @johngmyers #12830
• Rename fields to fit acronym conventions @johngmyers #12816
• Change DisableSubnetTags to tagSubnets @johngmyers #12832
• Promote alpha channel to stable @hakman #12807
• Add support for etcd v3.5.1 @hakman #12826
• Skip SCTP test on cilium clusters in k8s 1.24 as well @rifelpet #12838
• Fix enabling of AWS CCM @johngmyers #12837
• Fix upgrade tests with dest kOps version < 1.22 @johngmyers #12839
• Pin the aws-lb-controller scenario to k8s 1.21 @johngmyers #12840
• Change title of iam_roles.md to be specific to instance IAM roles @johngmyers #12845
• Route NAT64 to NAT Gateway in IPv6 private topology @johngmyers #12842
• Enable DNS64 in CoreDNS if IPv6 enabled @johngmyers #12766
• Invert sense of negative-option settings in v1alpha3 @johngmyers #12835
• Use AWS metadata to retrieve local-hostname in nodeup @bwagner5 #12844
• Fix e2e scenario tests @johngmyers #12847
• Open nodeport to IPv6 in e2e tests @johngmyers #12848
• Route NAT64 to NAT Gateway in IPv6 public topology @johngmyers #12843
• Use NAT64 instead of dual-stack for AWS EBS CSI driver @hakman #12850
• Reissue client keypairs on issuer change @johngmyers #12846
• Remove more non-configurable settings from v1alpha3 @johngmyers #12849
• Update IPv6 documentation @johngmyers #12852
• Revert "Open nodeport to IPv6 in e2e tests" @johngmyers #12854
• Support NodeLocalDNS on IPv6 clusters @johngmyers #12851
• Fix upgrade tests to kops version < 1.22 @johngmyers #12856
• Make requests and limits be *resource.Quantity @johngmyers #12857
• Stop skipping snapshot fields tests in EBS CSI e2e @rifelpet #12531
• Remove redundant evaluation of hostnameOverride @olemarkus #12858
• images: use k8s-staging-test-infra/gcb-docker-gcloud @spiffxp #12859
• images: fix invalid k8s-staging-test-infra/gcb-docker-gcloud tag @spiffxp #12861
• GCE: fix for metadata-proxy on cilium @justinsb #12866
• [Digital Ocean] Fix load balancer retry logic while retrieving ip @srikiz #12758
• Spotinst: Update spotinst/ocean-controller to v1.0.80 @liranp #12868
• Add GCP PD CSI driver addon @rifelpet #12812
• Fix area/provider/gcp GitHub label assignment @rifelpet #12871
• Add gofumpt scripts @hakman #12867
• Remove unused StorageBucketIam GCE task @rifelpet #12869
• Don't assign CIDRs to shared subnets @johngmyers #12863
• Add labels to GCE instance templates @rifelpet #12870
• Rename imageName to image in v1alpha3 @johngmyers #12872
• Use v1 certificate for snapshot-validation-service @olemarkus #12874
• tests gce: fix project arg on gsutil rm @justinsb #12875
• Remove support for CentOS 8 @johngmyers #12877
• If RBN, use IPv6 address instead of IPv4 @olemarkus #12878
• Support GSFS Terraform Managed Files @rifelpet #12121
• Rename bastionPublicName in v1alpha3 API @johngmyers #12876
• Remove support for old distros @hakman #12882
• Update k8s dependencies to v1.23.0-rc.0 @hakman #12883
• Add terraform integration test for dedicated apiserver nodes @rifelpet #12884
• Update k8s dependencies to v1.23.0-rc.1 @hakman #12888
• Allow setting cilium 1.11 @olemarkus #12887
• gce: Add network & subnet to toolbox dump @justinsb #12889
• GCE: Support kops-controller, including in gossip mode @justinsb #12742
• Use instance ID as node name when AWS CCM supports it @johngmyers #12862
• Update Go to v1.17.4 @hakman #12896
• enable connection-draining for aws classic lb @heybronson #12881
• Simplify Flatcar containerd exec command @pothos #12900
• pkg/apis/kops: Allow configuring dockerd --max-* upload and download concurrency and retry options. @anthonyrisinger #12320
• Upgrade node-cache to 1.21.3 @johngmyers #12904
• Fix external-dns service name @johngmyers #12893
• Fix error applying AWS CCM leader migration @hakman #12907
• Update k8s dependencies to v1.23.0 @hakman #12908
• Cleanup GCE loadbalancers created by k8s @zetaab #12894
• Set DNS PDB to a maxUnavailable percentage @heybronson #12915
• Update aws-load-balancer-controller to v2.3.1 @hakman #12920
• e2e: fix test flakes where we specify a non-schedulable zone @justinsb #12891
• Ignore InvalidAction errors when tagging IAM Instance Profiles @rifelpet #12629