This is a Beta release candidate and is NOT considered ready for deploying/upgrading ANY kubernetes clusters that you care about! Please do try out this version on your test clusters and help us to identify bugs! Please raise any bugs as issues on the project - thanks for your help!
With the v0.13.0 release of kube-aws we have decided to better align our release numbers with the release of kubernetes that they deploy, and so we have jumped over the kubernetes v1.12 release and our version 0.13.0 will deploy kubernetes v1.13 (presently v1.13.5). This release brings a number of changes related to the security and the stability of our kubernetes clusters and a number of other features.
Components
Kubernetes: 1.13.5
Etcd: 3.2.13
Important Upgrade Notes
- It is expected that you should be able to upgrade from existing kube-aws v0.12.x clusters to v0.13.0 release.
- If you use the Experimental Kiam feature you must regenerate your kiam credentials when upgrading to this release (you probably won't want to re-generate ALL of the certificates on an existing cluster though). We are now targeting Kiam release 3.2 which requires additional SANs on the server certificate
kiam-serverandlocalhost. Failure the update the certs will result in kiam-server and kiam-agents stuck in CrashLoopBackOff. - A number of Experimental Features are no longer available and will need to be removed from your
cluster.yamlfiles.- TLSBootstrap - is now active by default.
- PodPriority - is now active by default.
- PodSecurityPolicy - is now active by default (see warning below)
- NodeAuthorizer - is now active by default.
- PersistentVolumeClaimResize - is now active by default.
- DenyEscalatingExec - is deprecated and has been removed (please use PodSecurityPolicy)
- The previously unavailable admission controllers
EventRateLimitandExtendedResourceTolerationare now included and enabled by default. - If you do not have any existing PodSecurityPolicies in your cluster, kube-aws creates a permissive policy and binds it to all service accounts, authenticated users and nodes. Please create your own policies and then remove the permissive bindings by removing the ClusterRoleBinding
kube-aws:permissive-psp-cluster-wide.
Warning! If you do not have any existingPodSecurityPoliciesyou must ensure that they cover/allow all the workloads/pods that you want to run in your cluster, because once upgraded, no pods without a policy are allowed in the cluster! - If you have any scripts which connect to the kube-apiserver on port 8080 you will need to change them to use certificate based authentication via port 443.
Other Core Changes
- The kube-apiserver no longer listens on the insecure port
localhost:8080. All kube-aws services and scripts now communicate via the secure port (443) and use the admin certificate to authenticate. - The kubelets have authentication switched on and employ
Webhook authorizationto protect their services. We have added a number of RBAC objects to ensure scripts (such as cfn-signal) still have unauthenticated access to a kubelets/healthzendpoint. A number of kubelet settings are now set via a configuration file/etc/kubernetes/config/kubelet.yamlrather than by command-line switches. - We have removed
heapsterand enabledmetrics-serverby default. - We have moved to using CoreDNS instead of KubeDNS (but you can still select to use KubeDNS if you wish)
install-kube-systemhas undergone a refactoring which improves performance and adds flexibility in removing deprecated services. Theapply-kube-aws-pluginsservice has been rolled into theinstall-kube-systemand no longer exists.- A number of internal cluster components have seen version upgrades: -
- Calico/Typha v3.6.1
- Flannel v0.11.0
- Cluster AutoScaler v1.13.4
- Cluster Proportional Autoscaler 1.5.0
- CoreDNS 1.5.0
- KubeDNS 1.15.2
- Kiam 3.2
- DNSMasqMetrics 1.15.2
- Helm/Tiller v2.13.1
- Metrics Server v0.3.2
- Addon Resizer 2.1
Features
- #1480: Check stack existence via DescribeStacks(Thanks to @c-knowles)
- #1484: feat: add CF stackNameOverride in cluster.yaml(Thanks to @koen92)
- #1490: refactoring kube-aws / experimental IAM-based kubelet auth
- #1497: IAM Role name length checks need to account for StrictName usage...(Thanks to @davidmccormick)
- #1499: Change StackExists behaviour(Thanks to @davidmccormick)
- #1512: CA self-genereted cert: add the cn flag(Thanks to @sonant)
- #1514: 'AvailabilityZone' nodepool rolling strategy(Thanks to @davidmccormick)
- #1515: bug: Nodepool stack needs to use the nodepool logical name(Thanks to @davidmccormick)
- #1517: Etcd 3.3.10, compaction and defrag(Thanks to @davidmccormick)
- #1521: Move IAMRoleWorker definition from network stack to node pool stack(Thanks to @ktateish)
- #1527: Master: Add DnsMasq (node local resolver) command-line arguments/options(Thanks to @davidmccormick)
- #1531: feat: support for AWS mixed instances(Thanks to @koen92)
- #1539: Make dashboard replicas configurable(Thanks to @kylehodgetts)
- #1541: Adding the ability to set arbitrary feature gates to controller components(Thanks to @omar-nahhas)
- #1549: feat: Allow users to provide existing private keys(Thanks to @dominicgunn)
- #1555: Revert etcd upgrade(Thanks to @omar-nahhas)
- #1557: Make cluster-autoscaler resource request/limit configurable(Thanks to @Luke-Humphreys)
- #1564: Allowing adding arbitrary flags to kubernetes core components through plugins(Thanks to @omar-nahhas)
- #1571: Capture failures in decrypting assets(Thanks to @sgolightly)
- #1573: BUGFIX: kubeletOptions not applied on controller nodes.(Thanks to @omar-nahhas)
- #1575: Allow plugins to add CFN tags and outputs(Thanks to @Luke-Humphreys)
- #1576: Cluster-autoscaler plugin(Thanks to @Luke-Humphreys)
- #1582: Remove LaunchTemplateName from the LaunchTemplate(Thanks to @paalkr)
- #1583: Etcd Subnets not honoured on stack render.(Thanks to @dominicgunn)
- #1585: Enable detailed monitoring(Thanks to @paalkr)
- #1586: Custom APIServer SANs(Thanks to @dominicgunn)
- #1589: Release Prep to v0.13.x branch(Thanks to @davidmccormick)
Improvements
- #1536: Master: Bump default Kubernetes Dashboard version and add AllowSkipLogin option(Thanks to @kylehodgetts)
- #1567: Removing all references to the already removed kube-aws up/update commands(Thanks to @omar-nahhas)
Bug fixes
- #1495: Fix typos: additioanl->additional, creaes->creates(Thanks to @mooncak)
- #1505: Minor fixes in documentation(Thanks to @PabloCastellano)
- #1516: Fix for using StrictName to define a specific Controller Role Name(Thanks to @davidmccormick)
- #1522: Fix for kubelet startup on nodes when using TLS bootstrapping(Thanks to @davidmccormick)
- #1523: Fixes typo initizlie -> initialize(Thanks to @rbmrclo)
- #1526: Fix to escape % character in systemd config for raid0Mounts(Thanks to @cndbain)
- #1530: fix: Partial upgrades with stack name overrides(Thanks to @koen92)
- #1540: Fixing indentation error(Thanks to @omar-nahhas)
Documentation
- #1489: Fix some typos(Thanks to @AdamDang)
- #1560: Bring quickstart docs upto date (as of 2019)(Thanks to @kylehodgetts)
Other changes
- #1553: Fixing adding cf resources to etcd and networking stack - master(Thanks to @omar-nahhas)