What's new in v0.26.2
v0.26.2 is about getting the same answer twice. On one input, the 0.26.1 binary could print a different report on every run: the findings in another order, CO2 figures that differed in their last digits, another pair of calls in a chatty suggestion, another trace, endpoint and signature on a tied cross-trace slow finding. The daemon did the same across restarts. The cause was one pattern in 29 places: an order, a tie-break or a floating-point sum taken from a hash map, whose iteration order changes with every map the process builds. Fixing them also surfaced two undercounts of avoidable I/O, and one configuration mistake that only showed as randomness.
One input, one report
Batch runs now walk traces in the order their first span appears in the input, so the carbon pipeline sums its figures in a fixed order, and a service whose traces carry several regions reports the region of its first trace. Findings are sorted on a key that runs from type and severity down to the grouping, the timestamps, the service, the occurrence count and the code location, and the detectors that can still tie emit in input order. A chatty_service suggestion names its two most frequent calls with ties broken on the template, where it named whichever two a hash map listed first.
Against the 0.26.1 binary, ten runs of analyze on tests/fixtures/demo.json with a default_region gave 10 distinct JSON reports, 9 distinct SARIF files and 10 distinct terminal outputs, and ten runs of perf-sentinel demo gave 10 distinct outputs. With 0.26.2 every one of these gives a single output. On tests/fixtures/tied_findings.json, a new fixture that packs the ties, 0.26.1 gave 10 distinct outputs in each format and 0.26.2 one.
A tied slow finding names one trace
When several calls of a cross-trace slow_sql or slow_http finding tie on duration, the finding names the smallest trace ID, then the earliest timestamp, then the span ID. It took whichever tied trace a hash map listed last, and since the signature hashes the service and the endpoint, the signature moved with it. On four traces running the same 900 ms query from four endpoints, 0.26.1 produced 4 different signatures in 12 runs, so an acknowledgment matched some runs and not others. 0.26.2 names the same trace and signature every time. The daemon's windowed slow findings take the same tie-break for their service, endpoint and code location, and keep the most recent episode's trace so Explain can open it.
The daemon, restart after restart
/api/correlations, the correlations of /api/export/report, query correlations and the TUI panel sorted on confidence and co-occurrence count only. On 0.26.1, six restarts fed the same traces returned the same 53 correlations in six orders. They now continue on the source then the target, and the pairs evicted at max_tracked_pairs among tied ones go in pair order instead of map order. GET /api/acks and ack list list by signature, and the ack JSONL file is rewritten at startup in acknowledgment order, so it keeps its bytes across restarts. Findings the cross-batch slow window emits together come out sorted, and past max_events_per_trace the source endpoints of one batch are admitted in service then span ID order, so a span no longer resolves to its route on one run and to unknown on the next.
Avoidable I/O counts every grouping and every parameter set
The dedup that keeps one avoidable count per pattern was keyed on trace, template and endpoint. A template run in several groupings of one trace counted for one grouping only, and one template called with several parameter sets kept the largest redundant group instead of their sum: 3 calls with id = 1 and 4 with id = 2 counted 3 avoidable operations, not 5. The key now carries the grouping, redundant findings at one key add up, and the key keeps the larger of its N+1 figure and that sum. On tests/fixtures/tied_findings.json the avoidable count moves from 11 to 29 of 201 operations and the I/O waste ratio from 5.5% to 14.4%. The 20 other trace fixtures of the repository keep their figures.
A region key spelled twice
[green.service_regions] and [green.electricity_maps] region_map match case-insensitively and are lowercased at load, so two keys that differ only in case collapsed into one, and which value won followed a hash map. With "order-svc" = "eu-north-1" and "Order-Svc" = "us-east-1", 0.26.1 placed the service in eu-north-1 on 6 of 8 runs and in us-east-1 on the other 2, and its CO2 figures moved with it. 0.26.2 refuses that configuration at load with exit code 75 and an error that names both keys. Keys that agree on the value still load, and a region_map is only checked when a token enables the Electricity Maps section.
Upgrade impact
- Avoidable I/O and the figures built on it rise on traces that span several groupings or repeat one query with several parameter sets:
green_summary.avoidable_io_opsand its SQL and messaging parts,io_waste_ratio, avoidable CO2,perf_sentinel_avoidable_io_ops,perf_sentinel_service_avoidable_io_ops_totaland the canonical figures of periodic disclosures. They are not comparable across the upgrade for such traffic, and ananalyze --cirun whoseio_waste_ratio_maxrule passed near its threshold can now fail. - A configuration whose region keys differ only in case and map to different values fails to load. Keep one of the two keys before upgrading.
- Output order changes once. Findings, correlations and acks may list in another order than 0.26.1 printed on a given run, then stay put. A tied cross-trace slow finding may settle on another signature than the one acknowledged, in which case the acknowledgment needs renewing once.
perf-sentinel-coregains API:CorrelationEndpointandCodeLocationderivePartialOrdandOrd.- No finding appears or disappears, no detector verdict changes, no configuration key is added or removed, no route, metric name or wire format changes, the embedded reference data keeps its vintages, and MSRV stays 1.99.0.
Full detail in CHANGELOG.md.
Verifying this release
# Binary integrity via SLSA build provenance attestation (Build Level 2)
gh attestation verify perf-sentinel-linux-amd64 \
--repo robintra/perf-sentinel
# A periodic disclosure produced by this binary
perf-sentinel verify-hash --report perf-sentinel-report.json \
--expected-identity "https://github.com/robintra/perf-sentinel/.github/workflows/release.yml@refs/tags/v0.26.2" \
--expected-issuer "https://token.actions.githubusercontent.com" \
--verify-binary ./perf-sentinel-linux-amd64gh CLI 2.49 or newer required for gh attestation verify.