siderolabs/talos v1.2.0

on GitHub

Talos 1.2.0 (2022-09-01)

Welcome to the v1.2.0 release of Talos!

Please try out the release binaries and report any issues at
https://github.com/siderolabs/talos/issues.

Talos API access from Kubernetes

Talos now supports access to its API from within Kubernetes. It can be configured in the machine config as below:

machine:
  features:
    kubernetesTalosAPIAccess:
      enabled: true
      allowedRoles:
        - os:reader
      allowedKubernetesNamespaces:
        - kube-system

This feature introduces a new custom resource definition, serviceaccounts.talos.dev.
Creating custom resources of this type will provide credentials to access Talos API from within Kubernetes.

The new CLI subcommand talosctl inject serviceaccount can be used to configure Kubernetes manifests with Talos service accounts as below:

talosctl inject serviceaccount -f manifests.yaml > manifests-injected.yaml
kubectl apply -f manifests-injected.yaml

See documentation for more details.

Apply Config Patches

talosctl apply-config now supports patching the machine config file in memory before submitting it to the node.

Etcd Configuration

Configuration setting cluster.etcd.subnet is deprecated, but still supported.

Two new configuration settings are introduce to control precisely which subnet is used for etcd peer communication:

cluster:
  etcd:
    advertisedSubnets:
       - 10.0.0.0/24
    listenSubnets:
       - 10.0.0.0/24
       - 192.168.0.0/24

The advertisedSubnets setting is used to control which subnet is used for etcd peer communication, it will be advertised
by each peer for other peers to connect to. If advertiseSubnets is set, listenSubnets defaults to the same value, so that
etcd only listens on the same subnet as it advertises. Additional subnets can be configured in listenSubnets if needed.

Default behavior hasn't changed - if the advertisedSubnets is not set, Talos picks up the first available network address as
advertised address and etcd is configured to listen on all interfaces.

Note: most of the etcd configuration changes are accepted on the fly, but they are fully applied only after reboot.

Generating Talos secrets from PKI directory

It is now possible to generate a secrets bundle from a Kubernetes PKI directory (e.g. /etc/kubernetes/pki).

You can also specify a bootstrap token to be used in the secrets bundle.

This secrets bundle can then be used to generate a machine config.

This facilitates migrating clusters (e.g. created using kubeadm) to Talos.

talosctl gen secrets --kubernetes-bootstrap-token znzio1.1ifu15frz7jd59pv --from-kubernetes-pki /etc/kubernetes/pki
talosctl gen config --with-secrets secrets.yaml my-cluster https://172.20.0.1:6443

Kubernetes ControlPlane Components

Talos now run all Kubernetes Control Plane Components with the CRI default Seccomp Profile and other recommendations as described in
KEP-2568.

Kubelet Default Runtime Seccomp Profile

Talos now runs Kubelet with the CRI default Seccomp Profile enabled.
This can be disabled by setting .machine.kubelet.defaultRuntimeSeccompProfileEnabled to false.

This is not enabled automatically on upgrades, so upgrading to Talos v1.2 needs this to be explicitly enabled.

Kubernetes Control Plane labels and taints

Talos now defaults to node-role.kubernetes.io/control-plane label/taint.
On upgrades Talos now removes the node-role.kubernetes.io/master label/taint on control-plane nodes and replaces it with the node-role.kubernetes.io/control-plane label/taint.
Workloads that tolerate the old taints or having node selectors with the old labels will need to be updated.

Kubernetes Discovery Backend

Kubernetes cluster discovery backend is now disabled by default for new clusters.
This backend doesn't provide any benefits over the Discovery Service based backend, while it
causes issues for KubeSpan enabled clusters when control plane endpoint is KubeSpan-routed.

For air-gapped installations when the Discovery Service is not enabled, Kubernetes Discovery Backend can be enabled by applying
the following machine configuration patch:

cluster:
  discovery:
    registries:
      kubernetes:
        disabled: false

KubeSpan Kubernetes Network Advertisement

KubeSpan no longer by default advertises Kubernetes pod networks of the node over KubeSpan.
This means that CNI should handle encapsulation of pod-to-pod traffic into the node-to-node tunnel,
and node-to-node traffic will be handled by KubeSpan.
This provides better compatibility with popular CNIs like Calico and Cilium.

Old behavior can be restored by setting .machine.kubespan.advertiseKubernetesNetworks = true in the machine config.

MachineConfig .cluster.allowSchedulingOnMasters deprecated

The .cluster.allowSchedulingOnMasters is deprecated and replaced by .cluster.allowSchedulingOnControlPlanes.
The .cluster.allowSchedulingOnMasters will be removed in a future release of Talos.
If both .cluster.allowSchedulingOnMasters and .cluster.allowSchedulingOnControlPlanes are set to true, the .cluster.allowSchedulingOnControlPlanes will be used.

k8s.gcr.io mirror configuration

Talos now defaults to adding a registry mirror configuration in the machineconfig for k8s.gcr.io pointing to both registry.k8s.io and k8s.gcr.io unless overridden.
This is in line with the Kubernetes 1.25 release having the new registry.k8s.io registry endpoint.

This is only enabled by default on newly generated configurations and not on upgrades.
This can be enabled with a machine configuration as follows:

machine:
  registries:
    mirrors:
      k8s.gcr.io:
        endpoints:
          - https://registry.k8s.io
          - https://k8s.gcr.io

Network bridge support

Talos now supports configuring Linux bridges. It can be configured in the machine config like the following:

machine:
  network:
    interfaces:
      - interface: br0
        bridge:
          stp:
            enabled: true
          interfaces:
            - eth0
            - eth1

See documentation for more details.

VLAN support in cmdline arguments

Talos now supports dracut-style vlan kernel argument to allow
installing Talos Linux in networks where ports are not tagged
with a default VLAN:

vlan=eth1.5:eth1 ip=172.20.0.2::172.20.0.1:255.255.255.0::eth1.5:::::

NVIDIA GPU support promoted to beta

NVIDIA GPU support on Talos has been promoted to beta and SideroLabs now publishes the NVIDIA Open GPU Kernel Modules as a system extension making it easier to run GPU workloads on Talos. Refer to enabling NVIDIA GPU support docs here:

• https://www.talos.dev/v1.2/talos-guides/configuration/nvidia-gpu/
• https://www.talos.dev/v1.2/talos-guides/configuration/nvidia-gpu-proprietary/
• https://www.talos.dev/v1.2/talos-guides/configuration/nvidia-fabricmanager/

Packet Capture

Talos now supports capturing packets on a network interface with talosctl pcap command:

talosctl pcap --interface eth0

Seccomp Profiles

Talos now supports creating custom seccomp profiles on the host machine which in turn can be used by Kubernetes workloads.
It can be configured in the machine config as below:

machine:
  seccompProfiles:
    - name: audit.json
      value:
        defaultAction: SCMP_ACT_LOG
    - name: deny.json
      value: {"defaultAction":"SCMP_ACT_LOG"}

This profile data can be either configured as a YAML definition or as a JSON string.

The profiles are created on the host under /var/lib/kubelet/seccomp/profiles.

See documentation for more details.

Stable Default Hostname

Talos now generates the default hostname (when there is no explicitly specified hostname) for the nodes based on the
node id (e.g. talos-2gd-76y) instead of using the DHCP assigned IP address (e.g. talos-172-20-0-2).

This ensures that the node hostname is not changed when DHCP assigns a new IP to a node.

Please note: the stable hostname generation algorithm changed between v1.2.0-beta.0 and v1.2.0-beta.1, please take care when upgrading
from versions >= 1.2.0-alpha.1 to versions >= 1.2.0-beta.1 when using stable default hostname feature.

Strategic merge machine configuration patching

In addition to JSON (RFC6902) patches Talos now supports strategic merge patching.

For example, machine hostname can be set with the following patch:

machine:
  network:
    hostname: worker1

Patch format is detected automatically.

Variable substitution for URL query parameter in the talos.config kernel parameter

The kernel parameter talos.config can now substitute system information into placeholders inside its URL query values. This example shows all supported variables:

http://example.com/metadata?h=${hostname}&m=${mac}&s=${serial}&u=${uuid}

talosctl

--masters flag on talosctl cluster create is deprecated. Use --controlplanes instead.

Tracking progress of shutdown, reboot, reset and upgrade from CLI

talosctl subcommands shutdown, reboot, reset and upgrade now have a new flag --wait to
wait until the operation is completed, displaying information on the current status of each node.
A new --debug flag is added to these commands to get the kernel logs output from these nodes if the operation fails.

Component Updates

• Linux: 5.15.64
• Flannel 0.19.1
• containerd 1.6.8
• runc: v1.1.4
• Kubernetes: v1.25.0

Talos is built with Go 1.19.

Contributors

• Andrey Smirnov
• Noel Georgi
• Utku Ozdemir
• Artem Chernyshev
• Dmitriy Matrenichev
• Philipp Sauter
• Tim Jones
• Spencer Smith
• Davincible
• Eirik Askheim
• Steve Francis
• AMet
• Alex Wied
• Bermi Ferrer
• Branden Cash
• Charlie Haley
• Christoph Schmatzler
• Dennis Marttinen
• Eng Zer Jun
• Flightkick
• Florian Klink
• Gwyn
• Han Cen
• Larry Rosenman
• Markus Reiter
• Matthew Richardson
• Nico Berlee
• Rio Kierkels
• Robert Wunderer
• RyanSquared
• Serge Logvinov
• Seán C McCord
• Tommy Botten Jensen
• Trevor Sullivan
• hobyte
• nett_hier
• zebernst

Changes

Changes since v1.2.0-beta.2

Changes from siderolabs/discovery-client

Changes from siderolabs/extras

Changes from siderolabs/pkgs

Changes from siderolabs/tools

Changes from talos-systems/crypto

Changes from talos-systems/go-blockdevice

Changes from talos-systems/go-loadbalancer

Changes from talos-systems/grpc-proxy

Dependency Changes

• cloud.google.com/go/compute v1.6.1 -> v1.8.0
• github.com/BurntSushi/toml v1.1.0 -> v1.2.0
• github.com/aws/aws-sdk-go v1.44.24 -> v1.44.76
• github.com/containerd/containerd v1.6.4 -> v1.6.8
• github.com/containernetworking/cni v1.1.0 -> v1.1.2
• github.com/cosi-project/runtime 95d06feaf8b5 -> v0.1.1
• github.com/docker/docker v20.10.16 -> v20.10.17
• github.com/emicklei/dot v0.16.0 -> v1.0.0
• github.com/gertd/go-pluralize v0.2.1 new
• github.com/google/gopacket v1.1.19 new
• github.com/google/nftables a9775fb167d2 -> 2eca00135732
• github.com/hashicorp/go-getter v1.6.1 -> v1.6.2
• github.com/hashicorp/go-version v1.5.0 -> v1.6.0
• github.com/hetznercloud/hcloud-go v1.33.2 -> v1.35.2
• github.com/insomniacslk/dhcp 1ca156eafb9f -> 509691fd59ec
• github.com/jsimonetti/rtnetlink v1.2.0 -> v1.2.2
• github.com/martinlindhe/base36 v1.1.1 new
• github.com/mattn/go-isatty v0.0.14 -> v0.0.16
• github.com/packethost/packngo v0.24.0 -> v0.25.0
• github.com/prometheus/procfs v0.7.3 -> v0.8.0
• github.com/rivo/tview 9994674d60a8 -> 0e6b21a48e96
• github.com/siderolabs/discovery-client v0.1.1 new
• github.com/siderolabs/extras v1.1.0-1-g5800284 -> v1.2.0
• github.com/siderolabs/pkgs v1.1.0-8-gfa9a488 -> v1.2.0-8-g970860d
• github.com/siderolabs/tools v1.1.0-1-g134974c -> v1.2.0
• github.com/spf13/cobra v1.4.0 -> v1.5.0
• github.com/stretchr/testify v1.7.1 -> v1.8.0
• github.com/talos-systems/crypto v0.3.5 -> v0.3.6
• github.com/talos-systems/go-blockdevice v0.3.2 -> v0.3.4
• github.com/talos-systems/go-loadbalancer v0.1.2 -> v0.1.3
• github.com/talos-systems/grpc-proxy v0.3.0 -> v0.3.1
• github.com/u-root/u-root v0.8.0 -> v0.9.0
• github.com/vishvananda/netlink v1.2.0-beta -> v1.2.1-beta.2
• github.com/vmware-tanzu/sonobuoy v0.56.6 -> v0.56.9
• github.com/vmware/govmomi v0.28.0 -> v0.29.0
• go.uber.org/zap v1.21.0 -> v1.22.0
• golang.org/x/net 5463443f8c37 -> 3211cb980234
• golang.org/x/sync 0976fa681c29 -> 886fb9371eb4
• golang.org/x/sys bc2c85ada10a -> fbc7d0a398ab
• golang.org/x/term 065cf7ba2467 -> a9ba230a4035
• golang.org/x/time 583f2d630306 -> e5dcc9cfc0b9
• google.golang.org/grpc v1.46.2 -> v1.48.0
• google.golang.org/protobuf v1.28.0 -> v1.28.1
• gopkg.in/yaml.v3 496545a6307b -> v3.0.1
• inet.af/netaddr c74959edd3b6 -> 502d2d690317
• k8s.io/api v0.24.2 -> v0.25.0
• k8s.io/apimachinery v0.24.2 -> v0.25.0
• k8s.io/apiserver v0.24.2 -> v0.25.0
• k8s.io/client-go v0.24.2 -> v0.25.0
• k8s.io/component-base v0.24.2 -> v0.25.0
• k8s.io/cri-api v0.24.2 -> v0.25.0
• k8s.io/kubectl v0.24.2 -> v0.25.0
• k8s.io/kubelet v0.24.2 -> v0.25.0
• kernel.org/pub/linux/libs/security/libcap/cap v1.2.64 -> v1.2.65

Previous release can be found at v1.1.0

Images

ghcr.io/siderolabs/flannel:v0.19.1
ghcr.io/siderolabs/install-cni:v1.2.0
docker.io/coredns/coredns:1.9.3
gcr.io/etcd-development/etcd:v3.5.4
k8s.gcr.io/kube-apiserver:v1.25.0
k8s.gcr.io/kube-controller-manager:v1.25.0
k8s.gcr.io/kube-scheduler:v1.25.0
k8s.gcr.io/kube-proxy:v1.25.0
ghcr.io/siderolabs/kubelet:v1.25.0
ghcr.io/siderolabs/installer:v1.2.0
k8s.gcr.io/pause:3.6