robintra/perf-sentinel v0.25.0 on GitHub

What's new in v0.25.0

An incident recorded for one Kubernetes namespace held the findings of every namespace. On a fleet where each tenant runs the same service in its own namespace, a rollout that fired one alert per namespace recorded one incident per namespace, and each froze the same findings from every tenant. Those incidents were also heavy to list: GET /api/incidents sent every frozen finding of every incident on the page, up to 1000 each, to a client that often wanted only their number.

v0.25.0 freezes an incident's findings from its own namespace only, and lets a client list incidents without their findings.

Findings from the incident's own namespace

The freeze screened the findings ring by service and window alone, the incident's namespace being only a label. It now leaves out a finding whose grouping names a different k8s.namespace.name, at reception and in the settle pass, wherever that attribute sits in [detection] grouping_attributes. A finding that carries no such attribute is kept, since nothing places it elsewhere.

Nothing new to configure: the default grouping_attributes puts k8s.namespace.name first. A deployment grouped by another attribute alone freezes by service as before, as does an alert without a namespace, and the daemon warns at startup when [daemon.incidents] is enabled without that attribute among grouping_attributes. Incidents already recorded, in the ring or the archive, keep what they froze.

Incidents without their findings

GET /api/incidents takes a findings parameter. With findings=false each incident comes without its findings array and with finding_count in its place, every other field unchanged, by id as well as by page. The daemon builds these rows under the ring's read lock and counts the findings it leaves out rather than copying them.

The weight was not theoretical: a Grafana table reading /api/incidents?limit=50 for a count got thousands of findings on each refresh, which Grafana's Infinity parser inflated to several hundred megabytes until Grafana was OOM-killed.

Absent or true, the response is unchanged, so the archive, perf-sentinel query incidents and the Incidents tab of perf-sentinel query monitor see no difference, and an older daemon ignores the parameter and sends the findings. A malformed value answers 400 only after the key check, as include_toml does on GET /api/acks, and offset and limit now follow the same order, where a malformed one answered 400 before the key.

Upgrade impact

An incident with a namespace freezes fewer findings. Findings that name another k8s.namespace.name no longer land in it. Incidents recorded before the upgrade keep what they froze.
A new startup warning appears when [daemon.incidents] is enabled and k8s.namespace.name is missing from [detection] grouping_attributes. The daemon still starts.
A request to /api/incidents with a malformed offset or limit and no valid key now gets the key error, where it got a 400.
perf-sentinel-core gains two items: a namespace field on FindingsFilter, already #[non_exhaustive], and the config::K8S_NAMESPACE_ATTRIBUTE constant. Nothing is removed and no public type changes shape.
One query parameter is added and none is removed, no configuration key is added or removed, no metric or wire format changes, the acknowledgment signatures are untouched, the embedded reference data keeps its vintages, and MSRV stays 1.98.1.

Full detail in CHANGELOG.md.

Verifying this release

# Binary integrity via SLSA Build L3 attestation
gh attestation verify perf-sentinel-linux-amd64 \
  --repo robintra/perf-sentinel

# A periodic disclosure produced by this binary
perf-sentinel verify-hash --report perf-sentinel-report.json \
  --expected-identity "https://github.com/robintra/perf-sentinel/.github/workflows/release.yml@refs/tags/v0.25.0" \
  --expected-issuer "https://token.actions.githubusercontent.com" \
  --verify-binary ./perf-sentinel-linux-amd64

gh CLI 2.49 or newer required for gh attestation verify.