github rajsinghtech/garage-operator v0.8.0

2 hours ago

v0.8.0

Minor release: five new features (bucket website exposure, adopted bucketId, volumeAttributesClassName, pod extras, federated layout site roles), four fixes, a guard against Garage releases that crash with discovery, wider Garage version coverage in CI, and dependency/tooling updates. Changes since v0.7.12.

Highlights

Website exposure for buckets (#440, closes #434)

GarageBucket gains an optional spec.websiteExposure. The operator now creates an Ingress or a Gateway API HTTPRoute (exactly one, webhook-enforced) that routes a website-enabled bucket's hostname to the cluster's existing web API Service. No new proxy or per-bucket Service.

  • Resource is named <bucket>-website and created in the cluster's namespace.
  • Host defaults to <globalAlias><webApi.rootDomain>. An explicit host must match that pattern; an Ingress refuses hostnames that are neither the canonical host nor the global alias (it cannot rewrite Host), while an HTTPRoute can carry them via URLRewrite.
  • Outcomes are reported on a new WebsiteExposed condition and status.websiteExposure; the bucket itself never fails because of exposure problems (WaitingForAlias, GatewayAPIUnavailable, foreign-object name collision are surfaced, not fatal).
  • Cross-namespace exposures carry no owner reference; they are tracked by a label and removed on every deletion path (finalize, Retain, COSI retain), and bucket deletion is blocked (DeletionBlocked) if cleanup fails.
  • Adds RBAC for ingresses and httproutes (ClusterRole, namespace Role, and Helm chart) and a gatewayAPI.enabled chart value; adds sigs.k8s.io/gateway-api (types only).

spec.bucketId can be dropped after adoption (#431, closes #430)

After adopting an existing bucket, the resolved ID is remembered in status.bucketId, and spec.bucketId may then be removed (or left equal to the recorded value) without affecting the bucket. New guard rails:

  • Webhook rejects changing spec.bucketId to anything other than the recorded identity, and rejects a bucketId already claimed by another GarageBucket in the same cluster.
  • The controller fails closed on spec/status ID mismatch, releases the claim on a genuine 404, and refuses to delete a bucket another live GarageBucket still manages.
  • New Bucket ID print column.

Migration: set spec.bucketId → wait for kubectl get garagebucket <name> -o jsonpath='{.status.bucketId}' → remove the field.

volumeAttributesClassName for operator-managed volumes (#454, closes #445)

GarageCluster and GarageNode volume configs (metadata, data and each data.paths[] entry) gain an optional volumeAttributesClassName, applied to operator-managed PVCs so you can change IOPS/throughput through a Kubernetes VolumeAttributesClass without re-creating volumes.

  • A webhook rejects unsetting the class on a bound claim (the API server would allow it) and validates transitions; it warns if both the top-level and a data.paths[] class are set.
  • Each data.paths[] entry needs its own class; it does not inherit the top-level one.
  • If the API server rejects the field (e.g. an older cluster), the operator retries about every 5 minutes instead of on every reconcile; progress is reported through conditions and logs.
  • Requires a CSI driver that supports volume modification. Kubernetes Events for this are not emitted yet (follow-up).

Pod extras: initContainers, extraContainers, extraVolumes (#452, closes #441)

Add sidecars, extra init containers and extra volumes to the pod templates the operator generates. In v1beta2 they live under spec.storage, spec.gateway and spec.storage.nodeLocalPools[].podTemplate; in v1beta1 they are GarageCluster.spec.* and GarageNode.spec.*.

  • Uses schema-light wrapper types so the GarageCluster CRD stays well under etcd limits (about 1.06 MB); the webhook does the detailed validation.
  • Reserved-name rules prevent clashes with operator-owned containers and volumes.
  • Invalid extras block only that cluster's reconcile pass and leave the running workload untouched; the operator retries every 5 minutes.
  • Note: a v1beta1 client that omits the new fields clears them on update. Use v1beta2 when managing extras.
  • Fixes a bug where a sidecar could change the user the purge init container runs as.

Federated layout site roles: layoutManagement.siteRole (#453, closes #442)

In a multi-site federation, exactly one site should write the Garage layout. spec.layoutManagement.siteRole takes Writer or Follower; absent means Writer (no CRD default, so existing clusters are unchanged).

  • A Follower never writes layout (enforced by a client guard on all layout-write calls). The writer declares follower nodes as external GarageNodes; followers wait for the writer to add or remove roles, including on scale-down.
  • CEL rejects a follower without remoteClusters or with connectTo. The webhook rejects demoting a writer during a drain, rollout or factor migration, and warns on replication-setting changes at a follower (status AwaitingLayoutWriter, reason ReplicationChange).
  • The revert and skip-dead-nodes annotations are blocked on followers (removed with an event). New status.layoutWriter.
  • The same checks are enforced on v1beta1 so older clients cannot bypass them. Foreign-writer detection is planned for a later phase.

Garage compatibility

  • Discovery guard for Garage v2.3.0 and v2.4.0 (#456): those releases panic at start when any discovery section is configured; v2.4.1 fixes it. The webhook now warns when spec.image is one of them with spec.discovery.consul or .kubernetes. A new informational DiscoveryCompatible condition (and a Warning event on the transition) is derived from the version Garage reports, so it also covers digest-only images. It never drives Ready. Kubernetes discovery on v2.3.0 is treated as affected too (upstream reports only v2.4.0; this only widens a warning).
  • Native Kubernetes discovery is documented (#456): new how-to, a sample with the namespaced RBAC (garagenodes.deuxfleurs.fr get/list/create/update), skipCRD: true with a pre-applied CRD (config/samples/discovery/), and a webhook warning that the operator creates none of it. The operator's own peer connection is still the default; there is no raw-TOML escape hatch.
  • CI now covers the real range (#458): the whole e2e suite and the topology suites run the shipped default v2.4.1 (they ran v2.4.0 and v2.2.0), a floor lane runs the core cluster/bucket/key path on Garage v2.0.0, one lane runs v2.4.1 with kubernetes_discovery and RBAC, and a nightly informational workflow runs the same path on a build of Garage's main-v2 branch. The README no longer claims the default image is "the exact tested image".

Fixes

  • GarageKey.spec.importKey validation matches Garage v2.3+ (#457): inline credentials are accepted when the access key ID is at least 8 characters of [A-Za-z0-9-_.] and the secret is at least 16 graphic ASCII characters, so AWS/MinIO-style keys can be imported. Values outside the older GK+24 hex / 64 hex form get a warning that Garage v2.3+ is required. If Garage rejects the import (HTTP 400, e.g. on v2.2 or older) the key shows Ready=False, reason ImportKeyRejected, with Garage's own message and the running version (secret redacted). The CRD change is description-only. Stricter than before: inline secrets must now be at least 16 characters.
  • Idle buckets and keys stopped refreshing from Garage (#444, fixes #443): sub-second Garage timestamps were stored untruncated, so status never compared equal to the persisted value and the periodic drift requeue was skipped. Timestamps are now truncated to persisted precision and reconcilers always requeue after the drift interval. status.quotaUsage and bucket quota metrics now keep updating (one Admin API read per object per 5 minutes).
  • Missing GarageKey credentials recovered safely (#433): if the operator-owned Secret is missing while the remote key exists, credentials are recovered only from the pinned matching identity and an authoritative generated/imported source; otherwise it fails closed (nothing is fabricated or rotated). Also adds coverage for exact spec.bucketId handoff after a retained bucket owner is deleted.
  • Federated layout bootstrap race (#432): wait for local GarageNode identities and roles to settle before importing/enabling federation; local roles seen through federated status are ignored when their immutable cluster UID tag is already present.

Dependencies and tooling

  • controller-runtime 0.25.2 and sigs.k8s.io/gateway-api 1.6.2 (#446)
  • e2e: widen Auto→Manual ejection timeout to deflake CI (#449); version-matrix lanes and nightly main-v2 canary (#458)
  • k8s.io/* v0.37.0 → v0.37.1 (#437)
  • prometheus-operator monitoring API v0.94.0 → v0.94.1 (#435)
  • gomega v1.43.1 → v1.44.0 (#439)
  • golangci-lint → v2.14.0 (#438); kubectl → v1.37.1 (#436); Docker buildx → v0.37.2 (#448); golang:1.27 base image digest refresh (#447)

Upgrade notes

  • Garage v2.3.0 and v2.4.0 crash at start with any discovery configured: use v2.4.1 or newer (the built-in default) when enabling spec.discovery.
  • All API changes are additive. Re-apply CRDs (and upgrade the Helm chart) to pick up websiteExposure, the Bucket ID column, volumeAttributesClassName, the pod extras fields and layoutManagement.siteRole/status.layoutWriter (v1beta1 and v1beta2 both carry them).
  • volumeAttributesClassName needs a CSI driver and cluster with VolumeAttributesClass support. Pod extras and siteRole are no-ops until set.
  • RBAC gains networking.k8s.io/ingresses and gateway.networking.k8s.io/httproutes. Namespace-scoped installs get these in the namespace Role.
  • Rollback: older operators ignore websiteExposure. A cross-namespace exposure resource has no owner reference and would be orphaned if its bucket is deleted by an older operator; delete it manually in that case.

Contributors

Thank you to everyone who contributed to this release:

  • @littlejo made their first contribution with website exposure for buckets (#440, closes #434).
  • @ninuxio made their first contribution with the fix that keeps idle buckets and keys refreshing from Garage (#444, fixes #443), and also reported the issue.
  • @rajsinghtech for the remaining features, fixes, CI and docs.
  • @renovate[bot] for dependency updates.

Full Changelog: v0.7.12...v0.8.0

Don't miss a new garage-operator release

NewReleases is sending notifications on new releases.