kuik 3.0.0-beta.1 is the first beta of kuik v3, a rewrite of kuik from a clean tree. Every feature planned for 3.0 is in, except credentials from a cloud provider, but it is not stable yet: keep v2.3 for production. Features and breaking changes count from v2.3.1; fixes count from 3.0.0-alpha.6.
✨ Highlights
- New resources: kuik v3 routes, copies and monitors pod images with 3 new cluster-scoped resources:
ImageAlternative,ImageMirrorandImageMonitor. - Shared mirrors: an
ImageMirrorcan share its destination registry with other clusters: each cluster only creates, checks and deletes its own tags. - 3 processes: kuik runs as a webhook, a reconciler and a secret syncer, each with its own Deployment and 2 replicas by default.
- Paced registry reads: every background read of a registry is paced per host, to spare rate limits and quotas.
- No upgrade from v2 (yet): install v3 on a cluster without v2.
⚠️ Breaking changes
- From v2: v3 reads none of the v2 resources (
ImageSetMirror,ClusterImageSetMirror,ReplicatedImageSet,ClusterReplicatedImageSet,ClusterImageSetAvailability) nor the v2 chart values. Install v3 on a cluster without v2; a migration guide is planned after 3.0. (#670) - From
3.0.0-alpha.4or earlier: the chart requires Kubernetes 1.30 or later. (#752) - From
3.0.0-alpha.4or earlier: withserviceAccount.create: false, setsecretSyncer.serviceAccount.nameto an existing ServiceAccount other thandefault, or the chart refuses to render. (#752) - From
3.0.0-alpha.5or earlier:config.clusterIDis removed, and a release that still sets it fails to render. To keep the tags your mirrors copied, create the ConfigMapkuik-cluster-idwith the old value before upgrading, then removeconfig.clusterID(with--reuse-values, add--set config.clusterID=null), see Upgrading from a configured clusterID in the configuration docs. (#764, closes #727) - From
3.0.0-alpha.2or earlier: the processes log JSON instead of console text, with no chart value to switch back. (#703) - From
3.0.0-alpha.2or earlier: the rootreplicasvalue defaults to 2. (#701)
🚧 Known limits
Install
- Cloud credentials: credentials from a cloud provider (
auth.provider:aws,gcp,azure) are accepted but not used yet: kuik moves on to the next credential. Planned for 3.1. - One install per cluster: 2 secret syncers would refuse each other's writes.
- Reconciler first: the webhook admits pods unchanged until the reconciler has created
kuik-cluster-id, so never run withreconciler.replicas: 0. - Cluster ID: deleting the ConfigMap
kuik-cluster-id, or its namespace, generates a new cluster ID and orphans every tag already copied. Never copy it to another cluster that shares a destination: both would delete each other's tags. See the Cluster ID section of the configuration docs. - Backups: a restore tool such as Velero is refused the pull Secrets kuik manages; the syncer recreates them, and the restore ends partially failed.
Mirroring
- Tag deletion: a destination that never accepts tag deletion holds a deleted
ImageMirrorfor good: setcleanup.enabled: falsebefore deleting it, see Deleting an ImageMirror. - Harbor: Harbor refuses tag deletion through the registry API (any
DELETEby tag answers 400), socleanupdeletes nothing on a Harbor destination, and a deletedImageMirrorwithcleanup.enabled: truestays held by its finalizer. Setcleanup.enabled: falseon a Harbor destination and use Harbor's tag retention rules; support for tag deletion is requested in goharbor/harbor#24213.
Monitoring
- Drift digests: drift compares the digest the container runtime reports with the registry's: a runtime that reports a platform or config digest shows a tag as drifted, and a preloaded image is never checked for drift, see the ImageMonitor reference.
- Retention after a restart: an image whose last pod goes away while no reconciler runs (restart, leader change) is not retained by an
ImageMonitor.
Registries
- Node runtime: the
tlskey andinsecure: trueconfigure kuik only: configure the nodes' container runtime for the same hosts, or pods fail to pull.
📋 Changelog
Features
25 new features
Routing
- An
ImageAlternativelists repositories that hold the same images, and the webhook places the first candidate that answers, as a fallback (rewritePolicy: OnFailure, the default) or ahead of the original (Always). (#682) - Images match on whole path segments, with
repositoryfor one repository orrepositoryGroupfor every repository under a path, and digest-pinned images are routed too. (#680, #682) - An entry marked
unavailable: truedeclares a source gone, so an image from it is tried last. (#682) - The webhook records each decision on the pod in the
kuik.enix.io/rewrites,kuik.enix.io/conceded-rewritesandkuik.enix.io/no-alternativesannotations. (#682) - A candidate the background loops report failing is tried last (
webhook.availabilityCheck.demoteKnownFailures, on by default). (#751, #753) - The status of an
ImageAlternativereports the pods it rewrote, its active fallbacks and the images no candidate could serve. (#696)
Pull secrets
- A candidate's
auth.secretRefis copied as akuik-inject-<kind>-<name>pull Secret into the namespaces that need it, and added to the pod'simagePullSecrets. (#682, #701)
Mirroring
- An
ImageMirrorcopies the images of the pods it selects to a destination registry, digest unchanged, and routes to the copy unlessrewritePolicyisNone. (#682, #710) - Several clusters can share one destination: each copy carries a
_<clusterID>tag suffix, and each cluster only creates, checks and deletes its own tags. (#710) excludeImageskeeps images out of a mirror with globs such asghcr.io/foo-bar/**or**:latest. (#682, #710)driftPolicy(Ignore,Warn,Sync) decides what happens when an upstream tag moves after its copy. (#710)cleanupdeletes the destination tags no pod uses any more onceretentionelapsed (168h by default). (#710)- A deleted
ImageMirrorwaits while pods still run its copies, then deletes its tags. (#710)
Monitoring
- An
ImageMonitorchecks that the images of the pods it selects are still available upstream, and lists the failing ones instatus.unavailableImages. (#717, #732) - An image no pod uses any more stays monitored for
unusedImageRetention(168h by default), so a CronJob's image stays checked between runs. (#717) - With
driftDetection(on by default), it reports the tags whose upstream digest moved under the pods that run them, and the tags pods run under several digests. (#739) - With
monitorAlternatives: true, it also checks the alternativesImageAlternativeresources offer for those images. (#745)
Registries
- Background checks and copies are paced per host on fixed windows, set by
registriesin the global config. (#697) - Credentials are tried in the kubelet's order: the resource's
auth, the pod's pull secrets, the most specificfallbackAuthentry, then anonymous. (#680) - The
tlskey of the global config trusts a private CA (caSecretRef) or a self-signed certificate (insecureSkipVerify) per registry host, andinsecure: trueon an entry reaches a registry over plain HTTP. (#682, #710, #767, closes #750)
Configuration
- The global config file is validated whole and reloaded without a restart; a reload that does not validate keeps the previous config. (#680)
- kuik generates its cluster ID and keeps it in the ConfigMap
kuik-cluster-id. (#764, closes #727)
Observability
- Kubernetes events on pods and resources,
kuik_*Prometheus metrics for routing, copies, checks, anomalies and pacing, JSON logs, and conditions where onlyReadyisTruewhen all is well. (#696, #703, #710, #732)
Chart
Fixes
8 fixes
Routing
- A new webhook replica reports ready only once its routing index is built, so it never admits a pod unrouted. (#778)
Pull secrets
- The pull Secret of an
OnFailureresource is provided only in the namespaces itsnamespaceSelectorselects. (#777)
Mirroring
- A copy reads the original image first, before the alternatives of an
ImageAlternativewithrewritePolicy: Always. (#779) - A source that answers its check but fails the copy (a 429 on a blob, for instance) gives way to the next source instead of being retried at every window. (#779)
- A tag missing from the destination is copied again at the next copy window of its host, ahead of the initial copies. (#779)
- Adding or removing the cleanup finalizer no longer rewrites the spec, so
cleanup.retentionstays as written and GitOps tools see no drift. (#785)
Registries
- A CA bundle with a truncated certificate block is rejected, even when a complete block follows it. (#787)
Observability
kuik_registry_requests_totalis exported by the webhook and the reconciler, the processes that read registries. (#786)