robintra/perf-sentinel chart-v0.19.0

perf-sentinel chart v0.19.0

on GitHub

What's new in chart-v0.19.0

No template changes, no values.yaml key added or removed, and the default config.toml is byte-identical. appVersion moves to 0.19.0, so the image moves and pods roll, and the daemon that comes with it adds a second label to five metrics. That is breaking for alerts written against them without aggregating, and none of the alerts this chart ships is affected.

The daemon labels five metrics by grouping

perf_sentinel_findings_total, perf_sentinel_slow_duration_seconds and the three perf_sentinel_service_*_io_ops_total counters gain a grouping label next to service. Its value is the first attribute present from [detection] grouping_attributes, which defaults to k8s.namespace.name then service.namespace, so on Kubernetes it is the namespace the analysed workloads run in, not the one this chart installs into.

chart-v0.17.1   perf_sentinel_findings_total{severity="critical"}   1 series
chart-v0.18.0                                                      1 per service
chart-v0.19.0                                                      1 per (service, grouping)

Use sum by (service) to keep the 0.18.0 shape, sum() for the pre-0.18 one, or set per_grouping_labels = false under [daemon] in config.toml, which leaves the label declared but empty on every series and PromQL treats that as absent.

Cardinality is capped per daemon run on admitted (service, grouping) pairs, a second gate after the service caps: 512 on findings and the analysis-side I/O counters, 256 on the histogram, 4096 at ingest. A pair past its cap keeps its service and folds only its grouping into grouping="_other", so a sum by (service) still equals the 0.18.0 per-service series.

Why the label is not called namespace

serviceMonitor.honorLabels: true, shipped in 0.17.1, makes a label the daemon exposes win over the target label Prometheus Operator attaches. That is what carries service and now grouping through the operator's relabeling. It also means a daemon label called namespace would overwrite the operator's own, breaking the shipped dashboard's Daemon namespace variable and every namespace=~"$namespace" filter on it. grouping collides with nothing the operator attaches, which is why the label carries that name.

What it moves, and what it leaves alone

• No shipped alert changes. The PrometheusRule matches none of the five labeled metrics, and none of its rules selects on service or grouping.
• New counters, no alert on them. The daemon exposes perf_sentinel_analysis_grouping_overflow_total, perf_sentinel_slow_duration_grouping_overflow_total and perf_sentinel_service_io_ops_grouping_overflow_total, which count attributions folded into grouping="_other" rather than data lost. None ships as an alert, for the same reason PerfSentinelServiceCardinalityOverflow was removed in chart-v0.17.0: reaching a cap is a state the daemon works normally in.
• The checksum/config annotation does not move, unlike chart-v0.18.0. The default config.toml is unchanged, so the pod rolls on the image alone.
• per_grouping_labels is not a values.yaml key. It goes in the config.toml block of values.yaml, next to where per_service_labels would go, and both default to true in the daemon.

Upgrade impact

• Breaking for your own alerts and recording rules on the five metrics if they do not aggregate. An unaggregated alert on perf_sentinel_findings_total now fires per (service, grouping) pair, and its threshold is compared against a per-pair count rather than a total. Grep your rules before upgrading.
• Pods roll. image.tag is empty by default and falls back to .Chart.AppVersion, now 0.19.0, so the pod spec changes.
• An install already running per_service_labels = false needs per_grouping_labels = false too. The slow-span histogram's pre-warmed unlabeled series is minted at startup only when both knobs are off, otherwise it appears with the first slow span.
• Nothing here reaches an install without the operator. serviceMonitor.enabled: false renders no ServiceMonitor, and the label behavior above then depends on your own scrape configuration.

Install

The chart is published as an OCI artifact on GHCR, install it directly with no helm repo add step:

helm install perf-sentinel oci://ghcr.io/robintra/charts/perf-sentinel --version 0.19.0

Upgrade an existing release:

helm upgrade perf-sentinel oci://ghcr.io/robintra/charts/perf-sentinel --version 0.19.0

Read docs/HELM-DEPLOYMENT.md for the ServiceMonitor section, sizing and Ingress postures. The daemon image moves on this release, so the v0.19.0 binary notes describe what changes inside the pod.

If you are upgrading from chart-v0.17.1 or earlier, read the chart-v0.18.0 notes first: that release is where service appeared on two of these metrics. From chart-v0.16.0 or earlier, read the chart-v0.17.0 notes as well, which drop the shipped PrometheusRule from seven alerts to five and are breaking for prometheusRule.enabled: true.

Full Changelog: chart-v0.18.0...chart-v0.19.0