v1.9.0
The CNI v1.9.0 release will support higher pod density per node and also reduces the number of EC2 calls to create and attach more ENIs by leveraging the recent EC2 feature - Assigning prefixes to Amazon EC2 network interfaces. This feature is only supported on "nitro" based instances. Each secondary IP will be replaced by a (/28) prefix and IPAMD will derive a (/32) IP from these prefixes for pod IP allocation. Number of prefixes which can be attached to the ENI is still limited by the IP addresses per network interface per instance type- for instance a t3.medium can have 5 (/28)prefixes per ENI. Since the number of pods per node will increase, make sure the max pods are adjusted appropriately and keep in mind CPU and memory constraints will limit the number of pods that be deployed on a node. To help with the max pods computation we have added this helper script.
To enable the feature, set ENABLE_PREFIX_DELEGATION to true. This environment variable can be set to true or false while pods are running but kubelet maxPods has to be manually changed.
PS: This feature is available in all public regions and AWS GovCloud (US), with support in China (Beijing), and China (Ningxia) coming soon.
kubectl set env daemonset aws-node -n kube-system ENABLE_PREFIX_DELEGATION=true
With this feature we support WARM_PREFIX_TARGET and IPAMD will keep a free prefix in warm pool. We also support WARM_IP_TARGET and MINIMUM_IP_TARGET and IPAMD will allocate a new prefix if the existing prefixes are not sufficient to maintain the warm pool. This document talks about how these 3 warm targets impact the number of IPs available in the warm pool.
Since the IPs with this feature will be allocated from the prefixes, we won't be able to support downgrades. Also WARM_ENI_TARGET won't be supported with ENABLE_PREFIX_DELEGATION set to true.
Changes since v1.8.0:
- Enhancement - EC2 sdk model override (#1508, @jayanthvn)
- Enhancement - Prefix Delegation feature support (#1516, @jayanthvn)
- Enhancement - Header formatting for env variable (#1522, @jayanthvn)
- Enhancement - non-nitro instances init issues (#1527, @jayanthvn)
- Enhancement - Add metrics for total prefix count and ips used per cidr (#1530, @jayanthvn)
- Enhancement - Update documentation for PD (#1540, @jayanthvn)
- Enhancement - Update SDK Go version (#1544, @jayanthvn)
To apply this release:
kubectl apply -f https://raw.githubusercontent.com/aws/amazon-vpc-cni-k8s/v1.9.0/config/v1.9/aws-k8s-cni.yaml
Verify the update:
$ kubectl describe daemonset aws-node -n kube-system | grep Image | cut -d "/" -f 2
amazon-k8s-cni-init:v1.9.0
amazon-k8s-cni:v1.9.0