This version contains a critical update to etcd-manager: 1 year after creation (or first adopting etcd-manager), clusters will stop responding due to expiration of a TLS certificate. Upgrading kops to 1.18.0-beta.1 (or the latest versions of the 1.15, 1.16, 1.17 or 1.18 series) and running kops update
followed by a kops rolling-update
will fix the issue. Please see the advisory for the full details.
kops 1.18.0 beta.1 is the first beta in the 1.18 series for kops.
Please see the release notes for the full list of changes.
Significant changes
-
The default Docker version has been changed to 19.03.8.
-
Support for RHEL 8 and CentOS 8 has been added.
-
Support for Amazon Linux 2 has been improved and will work with the default Docker version.
-
containerd has been added and can be selected as an alternate container runtime for Kubernetes. Enable by using the
--container-runtime containerd
flag when creating a cluster or by settingspec.containerRuntime: containerd
. -
Rolling updates now support surging and parallelism within an instance group. For details see the documentation.
-
Cilium CNI can now use AWS networking natively through the AWS ENI IPAM mode. Kops can also run a Kubernetes cluster entirely without kube-proxy using Cilium's BPF NodePort implementation.
-
Cilium CNI can now use a dedicated etcd cluster managed by etcd-manager for synchronizing agent state instead of CRDs.
-
The Terraform target now supports Terraform 0.12 syntax (HCL2) by default. See the Required Actions item below.
-
New clusters in GCE are configured to run the metadata-proxy by default. The proxy runs as a DaemonSet and lands on nodes with the nodeLabel
cloud.google.com/metadata-proxy-ready: "true"
. If you want to enable metadata-proxy on an existing cluster/instance group, add that nodeLabel to your instancegroup specs (kops edit ig ...
) and runkops update cluster
. When the changes are applied, the proxy will roll out to those targeted nodes. -
GCE has a new flag:
--gce-service-account
. This takes the email of an existing GCP service account and launches the instances with it. This setting applies to the whole cluster (ie: it is not currently designed to support Instance Groups with different service accounts). If you do not specify a service account during cluster creation, the default compute service account will be used which matches the prior behavior. -
Google API client libraries updated from v0.beta to v1.
-
Kops does not support the "Legacy" etcd provider for Kubernetes versions 1.18 and higher. Such clusters will need to migrate to the default "Manager" etcd provider. To migrate, see the etcd migration documentation.
Breaking changes
-
Support for Docker versions 1.11, 1.12 and 1.13 has been removed because of the dockerproject.org shut down. Those affected must upgrade to a newer Docker version.
-
Terraform users on AWS may need to rename some resources in their state file in order to prepare for Terraform 0.12 support. See Required Actions below.
-
Support for the CoreOS OS distribution has been removed. Users should consider Flatcar as a replacement.
-
Support for the Debian 8 (Jessie) OS distribution has been removed.
-
The Docker
health-check
service is now disabled by default. It shouldn't be needed anymore, but it can still be enabled by settingspec.docker.healthCheck: true
. It is recommended to also check node-problem-detector and draino as replacements. See Required Actions below. -
Lyft CNI plugin default subnet tags changed from from
Type: pod
toKubernetesCluster: myclustername.mydns.io
. Subnets intended for use by the plugin will need to be tagged with this new tag and additional tag filters may need to be added to the cluster spec in order to achieve the desired set of subnets. -
Support for basic authentication has been disabled by default for Kubernetes 1.18 and will be removed in Kubernetes 1.19.
-
Support for static tokens has been disabled by default for Kubernetes 1.18 and later. To re-enable, see the Security Notes for Kubernetes. We intend to remove support entirely in a future kops version, so file an issue with your use case if you need this feature.
-
Support for Kubernetes versions prior to 1.9 has been removed.
-
Kubernetes 1.9 users will need to enable the PodPriority feature gate. See Required Actions below.
-
A controller is now used to apply labels to nodes. If you are not using AWS, GCE or OpenStack your (non-master) nodes may not have labels applied correctly.
-
The
kops/v1alpha1
API has been removed. Users ofkops replace
will need to supply v1alpha2 resources. -
Please see the notes in the 1.15 release about the apiGroup changing from kops to kops.k8s.io
Required Actions
-
Terraform users on AWS may need to rename resources in their terraform state file in order to support Terraform 0.12.
Terraform 0.12 no longer supports resource names starting with digits. In Kops, both the default route and additional VPC CIDR associations are affected. See #7957 for more information.- The default route was named
aws_route.0-0-0-0--0
and will now be namedaws_route.route-0-0-0-0--0
. - Additional CIDR blocks associated with a VPC were similarly named the hyphenated CIDR block with two hyphens for the
/
, for exampleaws_vpc_ipv4_cidr_block_association.10-1-0-0--16
. These will now be prefixed withcidr-
, for exampleaws_vpc_ipv4_cidr_block_association.cidr-10-1-0-0--16
.
To prevent downtime, follow these steps with the new version of Kops:
KOPS_FEATURE_FLAG=-Terraform-0.12 kops update cluster --target terraform ... # Use Terraform <0.12 terraform plan # Observe any aws_route or aws_vpc_ipv4_cidr_block_association resources being destroyed and recreated # Run these commands as necessary. The exact names may differ; use what is outputted by terraform plan terraform state mv aws_route.0-0-0-0--0 aws_route.route-0-0-0-0--0 terraform state mv aws_vpc_ipv4_cidr_block_association.10-1-0-0--16 aws_vpc_ipv4_cidr_block_association.cidr-10-1-0-0--16 terraform plan # Ensure these resources are no longer being destroyed and recreated terraform apply
Kops will now output Terraform 0.12 syntax with the normal workflow:
kops update cluster --target terraform ... # Use Terraform 0.12. This plan should be a no-op terraform plan
- The default route was named
-
Users that need the Docker
health-check
service will need to explicitly enable it:
kops edit cluster
# Add the following section
spec:
docker:
healthCheck: true
-
Kubernetes 1.9 users will need to enable the PodPriority feature gate. This is required for newer versions of Kops.
To enable the Pod priority feature, follow these steps:
kops edit cluster # Add the following section spec: kubelet: featureGates: PodPriority: "true"
-
If a custom Kops build was used on a cluster, a kops-controller Deployment may have been created that should get deleted.
Runkubectl -n kube-system delete deployment kops-controller
after upgrading to Kops 1.16.0-beta.1 or later.
Deprecations
-
Support for Kubernetes versions 1.9 and 1.10 are deprecated and will be removed in kops 1.19.
-
Support for Ubuntu 16.04 (Xenial) has been deprecated and will be removed in future versions of Kops.
All changes from v1.18.0-alpha.3 to v1.18.0-beta.1
- Add etcd-manager certificate expiration advisory @rifelpet #9030
- Treat NatGatewaysNotFound error as already-deleted @johngmyers #9052
- Allow cluster maintenance when channel is unavailable @johngmyers #9053
- Release notes for 1.18.0-alpha.3 @justinsb #9075
- Release notes for 1.17.0-beta.2 @justinsb #9073
- Disable TX checksum offload for Flannel VXLAN @hakman #9074
- Added support for configuring disable-attach-detach-reconcile-sync in… @andersosthus #9068
- Add advisory notice to readme and docs homepage @rifelpet #9083
- Revert "feat(openstack): propagate cloud labels to machines" @zetaab #9087
- kube-apiserver: healthcheck via sidecar container @justinsb #9069
- Include secondary protocol flag always @jacksontj #9008
- Fix port conflict on etcd-cilium vs dns-controller memberlist @justinsb #9097
- kube-apiserver-healthcheck: actually enable on 1.17 @justinsb #9098
- Update instance_groups.md @ranshn #9072
- Fix containerd image side-loading @hakman #9101
- Dont use terraform's file() for singleline strings in GCE metadata @rifelpet #9084
- Add documentation on gossip @olemarkus #9111
- upgrade to use cinder v3 api @zetaab #9113
- Fix zsh completion @olemarkus #9108
- Add unit test for util/pkg/hashing/hash.go @Hellcatlk #9114
- Spotinst: Allow users to disable the controller add-on @liranp #9091
- Fail cluster validation if too few nodes for ig's target size @johngmyers #9126
- Adding most recent version of kube-state-metrics - 1.9.5 @MoShitrit #9125
- PKI code cleanup @johngmyers #9106
- Return cluster validation failure if ASG missing @johngmyers #9118
- Add EC2 Instance LifeCycle label @atmosx #9121
- add some unit tests @q384566678 #8960
- Remove code for unsupported Kubernetes version @johngmyers #9134
- http download: set a timeout to avoid hangs @justinsb #9136
- Move CNI docs to their own files @olemarkus #9107
- Added Launch Template support for instance interruption behavior @tomesm,@rifelpet #9024
- DNS: Don't try to apply empty changesets @justinsb #8464
- Remove redundant menu item in the docs site @rifelpet #9144
- Remove Classic networking from docs @johngmyers #9142
- doc: Typo in docs/state.md @nvanheuverzwijn #9147
- Spotinst: Documentation @liranp #9139
- Map kube-apiserver service-account-jwks-uri flag @justinsb,@rifelpet #9133
- Don't put bastions in the utility subnets @johngmyers #9124
- Create golden image test for nodeup kube-apiserver @justinsb #8950
- Add unit test for func matchesElbTags @hs0210 #8989
- Remove support for reading legacy-format keypairs @johngmyers #9131
- Update alpha channels with May updates @MoShitrit #9155
- Add support for Kubenet with containerd @hakman #9104
- [Digital Ocean] Handle logic for kops edit/update cluster @srikiz #9116
- Move OS deprecations to deprecations section of relnotes @johngmyers #9093
- Add unit test case for pkg/k8sversion/version_test.go @Hellcatlk #9112
- Update OWNERS file @johngmyers #9105
- Minor doc fix. address is not valid to use, will cuase etcd faili… @granular-ryanbonham #9160
- GCE: don't rely on hostname being correct @justinsb,@rifelpet #9135
- Reduce test flakiness @johngmyers #9164
- Add unit test case for pkg/apis/kops/util/versions_test.go @Hellcatlk #9156
- Spotinst: New hybrid integration mode @liranp #7252
- Fix nodetask.File dependency on owner @johngmyers #9169
- Networking cleanup @olemarkus #9157
- Update DigitalOcean cloud-controller-manager to v0.1.24 @timoreimann #9179
- Update etcd-manager to 3.0.20200527 @justinsb #9184
- Use debian as default image for DO images @srikiz #9181
- Remove all versions of a file form the S3 bucket @hakman #9171
- Remove unused VFSScan @johngmyers #9174
- Remove loader support for nodeup tasks not used in models @johngmyers #9170
- Document etcd-manager backups retention settings @hakman #9187
- Add gjtempleton as reviewer @johngmyers #9183
- Fix nits for removal of S3 file versions @hakman #9188
- Remove support for CoreOS and Jessie @johngmyers #9065
- Update Bazel rules for Docker to v0.14.2 @hakman #9196
- Remove support for the legacy etcd provider as of k8s 1.18 @johngmyers #8826
- Add deprecation notice for legacy etcd provider to 1.17 relnotes @johngmyers #9201
- Add comment in OWNERS linking to test-infra OWNERS files @rifelpet #9202
- Fix repo packages not being installed @hakman #9203
- Allow listing versions for objects in the S3 bucket @hakman #9205
- Try validating multiple times before updating instancegroup @johngmyers #9165
- Use kubescheduler.config.k8s.io/v1beta1 for Kubernetes 1.19 @hakman #9204
- Update adding_a_feature.md with more modern example @johngmyers #9208
- Add example for delete secret @q384566678 #9198
- Upgrade docker/containerd/containeros hashes to SHA256 @johngmyers #9215
- Release notes for 1.16.3 @justinsb #9219
- Remove extraneous markdown files in pkg/apis @rifelpet #9220
- Release notes for 1.17.0 @justinsb #9222
- Remove unused file @johngmyers #9218
- Update set-version script to bump tag in Makefile @justinsb #9224
- Start release notes for kops 1.19 @justinsb #9223
- Use AWS SDK to fetch metadata @justinsb #9227
- S3 DeleteAllVersions: use pagination @justinsb #9228
- Bump compatibility matrix for kops 1.17 @johngmyers #9225
- Validation: MixedInstancePolicy need not override instance types @justinsb #9231
- GCE: fix typo @justinsb #9232
- Add packages hashes verification for containerd and Docker @hakman #9234
- Remove vsphere cloud provider @olemarkus #9177
- Update etcd-manager to 3.0.20200531 @hakman #9237
- Don't build site when docs are unchanged @hakman #9235
- Updating stable channel with May updates @MoShitrit #9212
- Upgrde amazon vpc cni to 1.6.2 @MoShitrit #9214
- Disable static tokens by default as of Kubernetes 1.18 @johngmyers #8850
Please see the release notes for the full list of changes.