cortexproject/cortex v1.18.0

Cortex v1.18.0

on GitHub

This release contains 230 contributions from 29 contributors. We also have 9 new contributors. Thank you all for the contributions!

Some notable changes release are:

• Experimental native histogram ingestion via -blocks-storage.tsdb.enable-native-histograms flag
• Support for filtering alerts on ListRules API
• Add query rejection mechanism to protect queries
• Introduce token bucket limiter to protect store gateway
• Implement ingester metadata API limits
• Remove -querier.query-store-for-labels-enabled flag
• Remove -querier.at-modifier-enabled flag
• Remove oltp_endpoint config
• ruler.evaluation-delay-duration mark as deprecated. Use ruler.query-offset
• -querier.max-outstanding-requests-per-tenant and -query-scheduler.max-outstanding-requests-per-tenant mark as deprecated. Use frontend.max-outstanding-requests-per-tenant

What's Changed

• [CHANGE] Ingester: Remove -querier.query-store-for-labels-enabled flag. Querying long-term store for labels is always enabled. #5984
• [CHANGE] Server: Instrument cortex_request_duration_seconds metric with native histogram. If native-histograms feature is enabled in monitoring Prometheus then the metric name needs to be updated in your dashboards. #6056
• [CHANGE] Distributor/Ingester: Change cortex_distributor_ingester_appends_total, cortex_distributor_ingester_append_failures_total, cortex_distributor_ingester_queries_total, and cortex_distributor_ingester_query_failures_total metrics to use the ingester ID instead of its IP as the label value. #6078
• [CHANGE] OTLP: Set AddMetricSuffixes to true to always enable metric name normalization. #6136
• [CHANGE] Querier: Deprecate and enable by default querier.ingester-metadata-streaming flag. #6147
• [CHANGE] QueryFrontend/QueryScheduler: Deprecate -querier.max-outstanding-requests-per-tenant and -query-scheduler.max-outstanding-requests-per-tenant flags. Use frontend.max-outstanding-requests-per-tenant instead. #6146
• [CHANGE] Ingesters: Enable 'snappy-block' compression on ingester clients by default. #6148
• [CHANGE] Ruler: Scheduling ruler.evaluation-delay-duration to be deprecated. Ruler will use the highest value between ruler.evaluation-delay-duration and ruler.query-offset #6149
• [CHANGE] Querier: Remove -querier.at-modifier-enabled flag. #6157
• [CHANGE] Tracing: Remove deprecated oltp_endpoint config entirely. #6158
• [CHANGE] Store Gateway: Enable store gateway zone stable shuffle sharding by default. #6161
• [FEATURE] Ingester/Distributor: Experimental: Enable native histogram ingestion via -blocks-storage.tsdb.enable-native-histograms flag. #5986 #6010 #6020
• [FEATURE] Querier: Enable querying native histogram chunks. #5944 #6031
• [FEATURE] Query Frontend: Support native histogram in query frontend response. #5996 #6043
• [FEATURE] Ruler: Support sending native histogram samples to Ingester. #6029
• [FEATURE] Ruler: Add support for filtering out alerts in ListRules API. #6011
• [FEATURE] Query Frontend: Added a query rejection mechanism to block resource-intensive queries. #6005
• [FEATURE] OTLP: Support ingesting OTLP exponential metrics as native histograms. #6071 #6135
• [FEATURE] Ingester: Add ingester.instance-limits.max-inflight-query-requests to allow limiting ingester concurrent queries. #6081
• [FEATURE] Distributor: Add validation.max-native-histogram-buckets to limit max number of bucket count. Distributor will try to automatically reduce histogram resolution until it is within the bucket limit or resolution cannot be reduced anymore. #6104
• [FEATURE] Store Gateway: Introduce token bucket limiter to enhance store gateway throttling. #6016
• [FEATURE] Ruler: Add support for query_offset field on RuleGroup and new ruler_query_offset per-tenant limit. #6085
• [ENHANCEMENT] Ruler: Add support to persist tokens in rulers. #5987
• [ENHANCEMENT] Query Frontend/Querier: Added store gateway postings touched count and touched size in Querier stats and log in Query Frontend. #5892
• [ENHANCEMENT] Query Frontend/Querier: Returns warnings on prometheus query responses. #5916
• [ENHANCEMENT] Ingester: Allowing to configure -blocks-storage.tsdb.head-compaction-interval flag up to 30 min and add a jitter on the first head compaction. #5919 #5928
• [ENHANCEMENT] Distributor: Added max_inflight_push_requests config to ingester client to protect distributor from OOMKilled. #5917
• [ENHANCEMENT] Distributor/Querier: Clean stale per-ingester metrics after ingester restarts. #5930
• [ENHANCEMENT] Distributor/Ring: Allow disabling detailed ring metrics by ring member. #5931
• [ENHANCEMENT] KV: Etcd Added etcd.ping-without-stream-allowed parameter to disable/enable PermitWithoutStream #5933
• [ENHANCEMENT] Ingester: Add a new limits_per_label_set limit. This limit functions similarly to max_series_per_metric, but allowing users to define the maximum number of series per LabelSet. #5950 #5993
• [ENHANCEMENT] Store Gateway: Log gRPC requests together with headers configured in http_request_headers_to_log. #5958
• [ENHANCEMENT] Ingester: Add a new experimental -ingester.labels-string-interning-enabled flag to enable string interning for metrics labels. #6057
• [ENHANCEMENT] Ingester: Add link to renew 10% of the ingesters tokens in the admin page. #6063
• [ENHANCEMENT] Ruler: Add support for filtering by state and health field on Rules API. #6040
• [ENHANCEMENT] Ruler: Add support for filtering by match field on Rules API. #6083
• [ENHANCEMENT] Distributor: Reduce memory usage when error volume is high. #6095
• [ENHANCEMENT] Compactor: Centralize metrics used by compactor and add user label to compactor metrics. #6096
• [ENHANCEMENT] Compactor: Add unique execution ID for each compaction cycle in log for easy debugging. #6097
• [ENHANCEMENT] Compactor: Differentiate retry and halt error and retry failed compaction only on retriable error. #6111
• [ENHANCEMENT] Ruler: Add support for filtering by state and health field on Rules API. #6040
• [ENHANCEMENT] Compactor: Split cleaner cycle for active and deleted tenants. #6112
• [ENHANCEMENT] Compactor: Introduce cleaner visit marker. #6113
• [ENHANCEMENT] Query Frontend: Add cortex_query_samples_total metric. #6142
• [ENHANCEMENT] Ingester: Implement metadata API limit. #6128
• [BUGFIX] Configsdb: Fix endline issue in db password. #5920
• [BUGFIX] Ingester: Fix user and type labels for the cortex_ingester_tsdb_head_samples_appended_total TSDB metric. #5952
• [BUGFIX] Querier: Enforce max query length check for /api/v1/series API even though ignoreMaxQueryLength is set to true. #6018
• [BUGFIX] Ingester: Fix issue with the minimize token generator where it was not taking in consideration the current ownership of an instance when generating extra tokens. #6062
• [BUGFIX] Scheduler: Fix user queue in scheduler that was not thread-safe. #6077 #6160
• [BUGFIX] Ingester: Include out-of-order head compaction when compacting TSDB head. #6108
• [BUGFIX] Ingester: Fix cortex_ingester_tsdb_mmap_chunks_total metric. #6134
• [BUGFIX] Query Frontend: Fix query rejection bug for metadata queries. #6143

New Contributors

• @Yaxhveer made their first contribution in #5920
• @KrisBuytaert made their first contribution in #5933
• @wilguo made their first contribution in #5935
• @rapphil made their first contribution in #5987
• @harshitasao made their first contribution in #6061
• @deradiri made their first contribution in #6099
• @shekeriev made their first contribution in #6125
• @klingerf made their first contribution in #6131
• @SungJin1212 made their first contribution in #6142

Full Changelog: v1.17.1...v1.18.0