kube-logging/logging-operator 6.8.0

on GitHub

What's Changed

New or updated images

Install with helm

helm install logging-operator oci://ghcr.io/kube-logging/helm-charts/logging-operator --version=6.8.0

Breaking changes

A duplicate parser name stops Fluent Bit from starting

Fluent Bit 5.1.0 makes a duplicate parser name fatal where 5.0 logged a warning and carried on.
A customParsers entry reusing a built-in name, such as json, docker, cri or
kube-custom, now crash-loops the DaemonSet. The parser the operator generates itself does not
collide. This is not in the Fluent Bit release notes; we found it while testing the bump.

Rename any colliding parser before upgrading, or pin the Fluent Bit image to 5.0.5.

Sidecar images older than custom-runner 1.0.0 no longer start

The config-reloader and buffer-metrics sidecars are now given an explicit -metrics-port, a flag
custom-runner gained in 1.0.0. Go rejects flags it does not know, so an older runner exits with
flag provided but not defined and the container crash-loops.

Move any configReloadImage or bufferVolumeMetricsImage pin to 6.8.0, or drop it and take
the default.

A syslog-ng image with only repository set now resolves to the pinned tag

syslogNGImage and metricsExporterImage were defaulted as whole structs, so setting just
repository, which is what you do to point at a registry mirror, left the tag empty and the
image resolved to :latest. Each field is defaulted on its own now, so the same spec pulls the
tag this release pins. If your mirror does not carry that tag, the pull fails.

Set tag explicitly if you were relying on :latest.

The syslog-ng metrics service overrides are applied

metricsService and bufferVolumeMetricsService have been on the CRD since syslog-ng support
landed, but nothing read them. They are merged into the generated Services now, so a value set
once and forgotten because it did nothing takes effect on upgrade.

Check both fields before upgrading.

Worth knowing

DB_Sync now reaches Fluent Bit. The operator wrote this setting as DB_Sync, which Fluent
Bit does not recognise, so it was dropped from the generated config. It renders as DB.sync now
and takes effect. The CRD field keeps its name and nothing needs editing, but the agent syncs its
position database as configured rather than not at all.

The syslog-ng config-reloader reports readiness. It has a readiness probe on the runner's
/readyz, which answers 503 naming the path when a configured file watch could not be
registered, the state where the reloader keeps running while silently never reloading again. A
pod that turns NotReady on upgrade was already failing to reload; the probe only makes it
visible.

IPv6

enabledIPv6 produced Services the cluster could not accept, and listeners that did not match
those Services. This release fixes three separate faults and changes what the flag does.

The operator now asks the API server which IP families it can allocate and names only those.
A cluster without an IPv6 range no longer has its Services rejected. syslog-ng also binds its
source for IPv6 under the same flag; previously only the Service changed, so an IPv6 primary
address pointed at a listener that was still IPv4 only and log ingestion stopped.

Kubernetes does not allow the primary IP family of an existing Service to change. Upgrading
therefore leaves an existing Service on its current primary family:

• Upgrading with enabledIPv6: true: the fluentd or syslog-ng Service keeps its IPv4
  primary and gains an IPv6 secondary. It is reachable over IPv6, but IPv4 stays first in
  ipFamilies, so Prometheus keeps scraping the IPv4 address.
• To get an IPv6 primary: delete the Service and let the operator recreate it.

  kubectl delete svc -n <control-namespace> <logging-name>-fluentd

  Expect a brief interruption. The Service comes back with a new cluster IP.
• New installs get an IPv6 primary where the cluster has an IPv6 range.

metrics.bind is a new field on the metrics type. It sets the listen address directly, which is
the supported way to get IPv6 metrics without forcing an IPv6 primary on the Service. The
syslog-ng metrics exporter has no listen address option, so it ignores the field.

Metrics

The syslog-ng config-reloader served its metrics on the runner's own default port while the
Service, the container port and the ServiceMonitor all named 9533, so nothing was ever scraped.
It listens on 9533 now.

sidecar_reloader_* series therefore appear for syslog-ng where there were none. A panel or
alert that has only ever seen an empty target starts receiving data, in particular
sidecar_reloader_config_reloader_last_reload_error.

Enhancements

• feat(syslog-ng): expose log_msg_size global option by @fdolsky321 in #2267
• feat(fluent-bit): expose storage.backlog.flush_on_shutdown option by @vyncint in #2282
• feat/support raw fluentd filter by @xtayfjpk in #2274
• feat: add helm chart parameter to enable raw fluentd filter or not by @xtayfjpk in #2289
• feat(chart): allow pinning the operator image by digest by @eumel8 in #2280
• build(e2e): run the linter on the e2e module, and fix what it reports by @vyncint in #2292
• refactor(e2e): remove dead code and correct the syslog-ng reloader image name by @vyncint in #2294
• refactor(e2e): move common/kind to internal/kind by @vyncint in #2297
• refactor(e2e): move common/cond to internal/wait and take plain values by @vyncint in #2298
• perf(e2e): load the operator images in one kind invocation by @vyncint in #2300
• refactor(e2e): add internal/fixture with the CR and tenancy builders by @vyncint in #2299
• feat(e2e): add internal/harness and move fluentbit-multitenant onto it by @vyncint in #2304
• fix: adopt custom-runner v1.0.0 by @csatib02 in #2311
• Share the configcheck abnormal-failure retry and cover it by @csatib02 in #2313
• Enable the modernize analyzer and apply it by @csatib02 in #2315

Dependency and image updates

• chore(deps): update dependency oj to v3.17.3 [security] by @renovate[bot] in #2256
• chore(deps): bump concurrent-ruby from 1.3.6 to 1.3.7 in /images/fluentd/outputs by @dependabot[bot] in #2257
• chore(deps): bump oj from 3.16.11 to 3.17.3 in /images/fluentd/outputs by @dependabot[bot] in #2258
• chore(deps): update module github.com/containerd/containerd to v1.7.33 [security] by @renovate[bot] in #2259
• chore(deps): bump faraday from 2.14.2 to 2.14.3 in /images/fluentd/outputs by @dependabot[bot] in #2262
• chore(deps): bump nokogiri from 1.19.3 to 1.19.4 in /images/fluentd/outputs by @dependabot[bot] in #2261
• chore(deps): bump concurrent-ruby from 1.3.6 to 1.3.7 in /images/fluentd/filters by @dependabot[bot] in #2260
• chore(deps): update dependency fluent-plugin-s3 to v1.8.5 [security] by @renovate[bot] in #2265
• chore(deps): bump fluentd from 1.18.0 to 1.19.3 in /images/fluentd/filters by @dependabot[bot] in #2266
• chore(deps): update module oras.land/oras-go/v2 to v2.6.1 [security] by @renovate[bot] in #2269
• chore(deps): bump excon from 1.3.0 to 1.5.0 in /images/fluentd/outputs by @dependabot[bot] in #2268
• chore(deps): bump oras.land/oras-go/v2 from 2.6.1 to 2.6.2 in /e2e by @dependabot[bot] in #2270
• chore(deps): update module golang.org/x/net to v0.56.0 [security] by @renovate[bot] in #2276
• chore(deps): update module google.golang.org/grpc to v1.82.1 [security] by @renovate[bot] in #2278
• chore(deps): update module golang.org/x/text to v0.39.0 [security] by @renovate[bot] in #2277
• chore(deps): update module go.opentelemetry.io/otel to v1.44.0 [security] by @renovate[bot] in #2281
• chore(deps): update module github.com/klauspost/compress to v1.18.7 [security] by @renovate[bot] in #2290
• build(deps): bump json from 2.20.0 to 2.21.2 in /images/fluentd/filters by @dependabot[bot] in #2312
• chore(deps): update all dependencies by @renovate[bot] in #2244

Bug fixes

• fix: support custom CA certificates for S3 outputs by @vyncint in #2272
• fix: enable dual-stack for Fluent Bit metrics services by @vyncint in #2273
• fix(ipv6): set IPv6 as primary address family when enabledIPv6 is set by @eumel8 in #2279
• fix(chart): support HostTailer namespaces by @vyncint in #2275
• fix(syslog-ng): keep default image tags when only the repository is set by @vyncint in #2283
• fix(fluentd): propagate dnsPolicy to the configcheck pod by @vyncint in #2285
• docs(plugins): fix dedot description from being incorrectly described as concat by @halkeye in #2284
• fix(e2e): bound kind CLI invocations with a timeout by @vyncint in #2288
• fix(configcheck): merge aggregator level configCheck field by field by @vyncint in #2286
• fix(e2e): derive the elasticsearch readiness budget from the test deadline by @vyncint in #2293
• fix(e2e): call LogProducer on the test goroutine by @vyncint in #2295
• fix(e2e): make coverage collection uniform and cover every cluster by @vyncint in #2296
• fix(e2e): remove the elasticsearch kill loop and five flake sources by @csatib02 in #2302
• fix(e2e): give every kind cluster its own kubeconfig by @vyncint in #2301
• fix(e2e): share the elasticsearch readiness budget between the waits by @vyncint in #2307
• fix(fluentd): propagate sidecarContainers to the configcheck pod by @pujitha24 in #2303
• Make enabledIPv6 work without breaking single-stack clusters by @csatib02 in #2310

New Contributors

• @fdolsky321 made their first contribution in #2267
• @vyncint made their first contribution in #2272
• @halkeye made their first contribution in #2284
• @pujitha24 made their first contribution in #2303

Full Changelog: 6.7.0...6.8.0