- [CHANGE] API: The experimental search endpoints now clamp
batch_sizeto 1000. #19731 - [CHANGE] Configuration: Default --web.external-url to https when --web.config.file enables TLS. #19241
- [CHANGE] PromQL: Subquery samples are now included in the query TotalSamples and TotalSamplesPerStep statistics, and SamplesRead now counts reads from subqueries with an @ modifier in range queries. #19165
- [CHANGE] Remote write: Stop exposing the deprecated
prometheus_remote_storage_metadata_total,prometheus_remote_storage_metadata_failed_total,prometheus_remote_storage_metadata_retried_total, andprometheus_remote_storage_metadata_bytes_totalmetrics for Remote Write 2.0 queues. #20080 - [CHANGE] Remote write receiver: Count Remote Write 2.x samples and histograms ingested without metadata in
prometheus_api_remote_write_without_metadata_appended_samples_totaleven whenmetadata-wal-recordsis disabled. #19812 - [CHANGE] TSDB: Reject exemplars whose label names are not unique, instead of storing them and serving a duplicate JSON key from
/api/v1/query_exemplars. #19896 - [FEATURE] API: Allow overriding PromQL Start Timestamp processing on a per-query basis using the
X-Prometheus-Use-Start-TimestampsHTTP header. #19066 - [FEATURE] Agent: Support metadata-wal-records in Prometheus Agent mode. #19684
- [FEATURE] PromQL: Add experimental
integral()function for computing integral-over-time of range vectors behind the--enable-feature=promql-experimental-functionsflag. #14440 - [FEATURE] PromQL: Add experimental query cost estimation, actual cost reporting, and configurable execution limits, enabled with
--enable-feature=query-cost. #20028 - [FEATURE] PromQL: Smoothed and Anchored vector selectors are now stable and always-on. #19697
- [FEATURE] PromQL: Use start timestamps to detect counter resets when float timeseries are queried using extended range selector modifiers (
anchored,smoothed). This feature is hidden behinduse-start-timestampsfeature flag. #18966 - [FEATURE] Promtool: print test filename during promtool tests #18271
- [FEATURE] PromQL: Allow unquoted Unicode letters and dots behind a feature flag #20162
- [ENHANCEMENT] Search API: Warn when participating remote-read storage does not support search. #19992
- [ENHANCEMENT] Remote Read: Add native histogram prometheus_remote_read_handler_gate_wait_duration_seconds to measure concurrency-gate wait duration, including requests canceled while waiting. #18355
- [ENHANCEMENT] Remote Write: Add experimental flag
convert_nhcb_to_classicto convert NHCB to classic histograms for external systems that dont support NHCBs. #17205 - [ENHANCEMENT] promtool: Add
--allow-incompatible-block-durationso users can specify any max-block-duration instead of being restricted to the default Prometheus block durations. #19363 - [PERF] Remote write: Cache repeated OTLP histogram metric names. #19717
- [PERF] OpenMetrics 2.0 parser: Reuse classic histogram and summary expansion buffers across scrapes. #19907
- [PERF] PromQL: Decode only histogram sum/count (not buckets) for timestamp(), start_timestamp() and absent(). #19738
- [PERF] PromQL: Only select the fixed window for subqueries with an
@modifier. #19863 - [PERF] Remote write / TSDB: Deduplicate per-series metadata structs in memory with
unique.Handle. #20040 - [PERF] Scrape: Avoid some copies of scrape response bodies. #19394
- [PERF] TSDB: Improve XOR2 compression and append performance by selecting value windows based on encoding cost. #19962
- [PERF] TSDB: Reduce memory used writing and reading checkpoint metadata records. #19712
- [PERF] Scrape: Convert classic histograms to NHCB directly in the OpenMetrics 2.0 parser without wrapping in
NHCBParser. #19880 - [PERF] Scrape: Reduce OpenMetrics 2.0 parser allocations and CPU overhead by lazily constructing composite series and exemplar labels. #19895
- [BUGFIX] Remote read: Abort failed streaming responses without writing duplicate headers or appending plain-text errors to the protobuf stream #19749
- [BUGFIX] TSDB: Fix loss of out-of-order samples when a WAL checkpoint drops a series record that the out-of-order write-behind log still needs #19737
- [BUGFIX] TSDB: Fix queries that seek within a head chunk returning samples committed after the query started #19854
- [BUGFIX] TSDB: Preserve mapped chunks when WAL replay merges multiple references for the same series, preventing missing out-of-order samples and duplicate in-order chunks after restart #19805
- [BUGFIX] Fix auto-reload logic to reload after reverting a failed configuration change #19838
- [BUGFIX] API: Do not match metric family metadata for doubled _total and _info suffixes in experimental search API. #19910
- [BUGFIX] API: Reject
timeoutandlookback_deltavalues that overflow int64 instead of turning them into a negative duration #19658 - [BUGFIX] API: Resolve metric family metadata for suffixed metric names in search results, in experimental search API. #19824
- [BUGFIX] AWS SD: Describe each ElastiCache cluster once instead of twice for clusters that are not replication group members. #19434
- [BUGFIX] Build: Allow installing promtool with version-suffixed go install commands #19754
- [BUGFIX] Discovery: Fix a panic in Azure service discovery when a VM tag has a null value. #19755
- [BUGFIX] Federation: fix corruption of float native histograms. #19679
- [BUGFIX] Fix ts_of_last_over_time for samples before the Unix epoch. #19828
- [BUGFIX] Histograms: Fix ConvertNHCBToClassic inflating classic bucket counts for integer native histograms whose spans have gaps. #19659
- [BUGFIX] Logging: Preserve rule and TSDB block values in JSON logs. #19572
- [BUGFIX] OTLP receiver: Drop exemplars with timestamps more than 10 minutes in the future instead of storing them and blocking subsequent exemplars for that series. #19812
- [BUGFIX] OTLP: Count only out-of-order exemplars in
prometheus_api_otlp_out_of_order_exemplars_total, not every rejected exemplar. #20030 - [BUGFIX] OTLP: Do not drop exemplars that have a
trace_idorspan_idfiltered attribute in addition to a trace or span ID. The trace context now takes precedence over the colliding attribute, as required by the OpenTelemetry specification. #20004 - [BUGFIX] OTLP: Do not emit
le="+Inf"twice when a histogram's explicit bounds end at+Inf, and warn about the bound. #19425 - [BUGFIX] OTLP: Stop dropping exemplars that arrive out of timestamp order within a data point. #20029
- [BUGFIX] OpenMetrics 2.0: Attach histogram exemplars to matching classic
_bucketseries and preserve them when scraping both native and classic histograms. #20020 - [BUGFIX] PromQL: Avoid intermediate overflow in
rad()anddeg()conversions. #19901 - [BUGFIX] PromQL: Consistently ignore and drop
__type__and__unit__labels inwithoutaggregations,absent(),info(), and binary operator fill matching. #19986 - [BUGFIX] PromQL: Do not panic when printing the partial AST returned alongside a parse error. #19797
- [BUGFIX] PromQL: Drop type and unit in binary operations so they are transparent to users, similar to how name is handled. #18680
- [BUGFIX] PromQL: Fix
==,!=,changes()andcount_valueson native histograms giving wrong results insidehistogram_count(),histogram_sum()andhistogram_avg(). #20024 - [BUGFIX] PromQL: Fix histogram_fraction returning wrong results for native histograms with custom buckets when a bucket boundary is exactly 0. #19657
- [BUGFIX] PromQL: Fix histogram_quantile and histogram_quantiles interpolating within the wrong bucket for native histograms whose sum is NaN. #19973
- [BUGFIX] PromQL: Fix lexing of subqueries whose range is a duration keyword expression, e.g.
foo[range():1s]. #19729 - [BUGFIX] PromQL: Fix warning and error locations for duration expressions
min_ofandmax_of. #19742 - [BUGFIX] PromQL: Ignore NaN classic histogram bucket boundaries in
histogram_quantile,histogram_quantilesandhistogram_fractionand return correct value instead of NaN if there are normal buckets. #20057 - [BUGFIX] PromQL: Preserve infinite values at exact quantile ranks. #19899
- [BUGFIX] PromQL: Preserve original operand sides for experimental fill modifiers with group_right matching. #19939
- [BUGFIX] PromQL: Preserve subquery step expressions when printing queries after preprocessing. #19743
- [BUGFIX] PromQL: Range vector functions no longer raise a spurious "vector cannot contain metrics with the same labelset" error when series have different metric names but the same remaining labels, as long as their timestamps do not overlap within the evaluated range window #18631
- [BUGFIX] PromQL: do not count subquery materialized samples in total samples statistics #19873
- [BUGFIX] Remote write: Prevent future-dated exemplars from scaling shards down to the minimum. #19866
- [BUGFIX] Agent: Prevent future sent timestamps from advancing WAL truncation beyond the minimum retention window. #19866
- [BUGFIX] Rules: Restore firing alerts with for durations shorter than --rules.alert.for-grace-period after a restart. #19882
- [BUGFIX] Scrape: fix failing scrape when float histogram is emitted over protobuf with count, zero count and buckets all zero, for example a result of recording
rate(x[1m])over constant histogram. #19682 - [BUGFIX] Search API: Return the lexically smallest label values when
limitis set on/api/v1/search/label_values, instead of an arbitrary subset #19694 - [BUGFIX] TSDB: Fix WAL watcher permanently skipping checkpoint series garbage collection when lagging behind head truncation. #20041
- [BUGFIX] Search API: Avoid panics when the requested time range has no participating storage. #19992
- [BUGFIX] TSDB: Fix a race in selected-series and stale-series compaction where a sample committed after the block was written but before the series was evicted from the head could be silently lost. The fix no longer depends on isolation being enabled. #19666
- [BUGFIX] TSDB: Fix a rare case where a sample could be lost during selected-series or stale-series compaction if an unrelated write to the same series ran cleanup that erased the evidence needed to protect it. #19672
- [BUGFIX] TSDB: Fix spurious "unknown series references" warnings during WAL replay after a restart, when the only blocks on disk were produced by selected-series or stale-series compaction. #19734
- [BUGFIX] TSDB: Increment
prometheus_tsdb_wal_corruptions_totalwhen WAL corruption is detected during checkpoint creation, so the metric reflects corruption found at compaction time rather than only after a restart-triggered repair. #19858 - [BUGFIX] TSDB: Preserve series identities needed to recover exemplars after compaction and restart. #19940
- [BUGFIX] TSDB: Reject native histograms whose span offsets overflow the int32 bucket index instead of storing them with corrupted buckets. #20034
- [BUGFIX] TSDB: fix no passing metadata-wal-records to TSDB from main feature flag. #19678
- [BUGFIX] Templates: Preserve IPv6 addresses with dotted zone identifiers in
stripDomain. #19908 - [BUGFIX] UI: Don't automatically open an autocomplete dropdown after closing a function call body or aggregation operator with a closing parenthesis. #19825
- [BUGFIX] Web: Fix HTTP/2 negotiation silently falling back to HTTP/1.1 under Go 1.27+ when TLS is enabled with
http2: true. The server now correctly advertisesh2via ALPN and multiplexes concurrent requests over a single connection. #19848 - [BUGFIX] mixin: Fix double-counting of failed samples in the
PrometheusRemoteStorageFailuresalert denominator. #19877 - [BUGFIX] promtool: Honor
--enable-featurewhen evaluating rules intest rules. #20046 - [BUGFIX] Scrape: Ensure UTF-8 works with the experimental OpenMetrics 2.0 #19902
- [BUGFIX] Scrape: Reject sum-only classic histograms during NHCB conversion #19315