aws/eks-anywhere v0.17.0

on GitHub

Supported OS version details

Note: We have updated the image-builder docs to include the latest enhancements. Please refer to the image-builder docs for more details.

Added

• Add support for AWS CodeCommit repositories in FluxConfig with git configuration #4290
• Add new information to the EKS Anywhere Cluster status #5628:
  • Add the ControlPlaneInitialized, ControlPlaneReady, DefaultCNIConfigured, WorkersReady, and Ready conditions.
  • Add the observedGeneration field.
  • Add the failureReason field.
• Add support for different machine templates for control plane, etcd, and worker node in vSphere provider #4255
• Add support for different machine templates for control plane, etcd, and worker node in Cloudstack provider #6291
• Add support for Kubernetes version 1.25, 1.26, and 1.27 to CloudStack provider #6167
• Add bootstrap cluster backup in the event of cluster upgrade error #6086
• Add support for organizing virtual machines into categories with the Nutanix provider #6014
• Add support for configuring egressMasqueradeInterfaces option in Cilium CNI via EKS Anywhere cluster spec #6018
• Add support for a flag for create and upgrade cluster to skip the validation --skip-validations=vsphere-user-privilege
• Add support for upgrading control plane nodes separately from worker nodes for vSphere, Nutanix, Snow, and Cloudstack providers #6180
• Add preflight validation to prevent skip eks-a minor version upgrades #5688
• Add preflight check to block using kindnetd CNI in all providers except Docker #6097

Upgraded

• Cluster API Provider vSphere: v1.6.1 to v1.7.0
• Cluster API Provider CloudStack: v0.4.9-rc5 to v0.4.9-rc6
• Cluster API Provider Nutanix: v1.2.1 to v1.2.3

Cilium Upgrades

• Cilium: v1.11.15 to v1.12.11 (changelog)

  Note: If you are using the vSphere provider with the Redhat OS family, there is a known issue with VMWare and the new Cilium version that only affects our Redhat variants. To prevent this from affecting your upgrade from EKS Anywhere v0.16 to v0.17, we are adding a temporary daemonset to disable UDP offloading on the nodes before upgrading Cilium. After your cluster is upgraded, the daemonset will be deleted. This note is strictly informational as this change requires no additional effort from the user.

Changed

• Change the default node startup timeout from 10m to 20m in Bare Metal provider #5942
• EKS Anywhere now fails on pre-flights if a user does not have required permissions. #5865
• eksaVersion field in the cluster spec is added for better representing CLI version and dependencies in EKS-A cluster #5847
• vSphere datacenter insecure and thumbprint is now mutable for upgrades when using full lifecycle API 6143

Fixed

• Fix cluster creation failure when the <Provider>DatacenterConfig is missing apiVersion field #6096
• Allow registry mirror configurations to be mutable for Bottlerocket OS #2336
• Patch an issue where mutable fields in the EKS Anywhere CloudStack API failed to trigger upgrades #5910
• image builder: Fix runtime issue with git in image-builder v0.16.2 binary #2360
• Bare Metal: Fix issue where metadata requests that return non-200 responses were incorrectly treated as OK #2256

Known Issues

• Upgrading Docker clusters from previous versions of EKS Anywhere may not work on Linux hosts due to an issue in the Cilium 1.11 to 1.12 upgrade. Docker clusters is meant solely for testing and not recommended or support for production use cases. There is currently no fixed planned.
• If you are installing EKS Anywhere Packages, Kubernetes versions 1.23-1.25 are incompatible with Kubernetes versions 1.26-1.27 due to an API difference. This means that you may not have worker nodes on Kubernetes version <= 1.25 when the control plane nodes are on Kubernetes version >= 1.26. Therefore, if you are upgrading your control plane nodes to 1.26, you must upgrade all nodes to 1.26 to avoid failures.
• There is a known bug with systemd >= 249 and all versions of Cilium. This is currently known to only affect Ubuntu 22.04. This will be fixed in future versions of EKS Anywhere. To work around this issue, run one of the follow options on all nodes.

Option A

# Does not persist across reboots.
sudo ip rule add from all fwmark 0x200/0xf00 lookup 2004 pref 9  
sudo ip rule add from all fwmark 0xa00/0xf00 lookup 2005 pref 10  
sudo ip rule add from all lookup local pref 100  

Option B

# Does persist across reboots.
# Add these values /etc/systemd/networkd.conf
[Network]
ManageForeignRoutes=no
ManageForeignRoutingPolicyRules=no

Deprecated

• The bundlesRef field in the cluster spec is now deprecated in favor of the new eksaVersion field. This field will be deprecated in three versions.

Removed

• Installing vSphere CSI Driver as part of vSphere cluster creation. For more information on how to self-install the driver refer to the documentation here

⚠️ Breaking changes

• CLI: --force-cleanup has been removed from create cluster, upgrade cluster and delete cluster commands. For more information on how to troubleshoot issues with the bootstrap cluster refer to the troubleshooting guide (1 and 2). #6384