VictoriaMetrics/VictoriaMetrics v1.140.0

on GitHub

v1.140.0

Released at 2026-04-10

Update Note 1: vmsingle and vmselect in VictoriaMetrics cluster: CSV export (/api/v1/export/csv) now adds a header row as the first line of the response, so existing CSV-processing scripts may need to skip this header. See #10666.

• SECURITY: upgrade Go builder from Go1.26.1 to Go1.26.2. See the list of issues addressed in Go1.26.2.

• FEATURE: vmagent: add per-URL -remoteWrite.disableMetadata flag to disable metadata sending for specific remote storage URLs. See #10711. Thanks to @evkuzin for the contribution.

• FEATURE: vmagent: add profile option to ec2_sd_configs for loading credentials from named AWS profiles in ~/.aws/credentials and ~/.aws/config, including source_profile chaining and role_arn resolution. See ec2_sd_configs docs. Issue #1685. Thanks to @andriibeee for the contribution.

• FEATURE: introduce vm_filestream_fsync_duration_seconds_total and vm_filestream_fsync_calls_total metrics, which can be used for detecting slow storage if it cannot keep up with the current data ingestion rate. See #10432. Thanks to @mehrdadbn9 for the contribution.

• FEATURE: vmctl: add dedicated thanos mode for migrating data from Thanos. This mode supports both raw and downsampled Thanos blocks, including all aggregate types (count, sum, min, max, counter). Each aggregate is imported as a separate metric with resolution and aggregate type suffixes (e.g., metric_name:5m:count). The new mode uses --thanos-* prefixed flags: --thanos-snapshot, --thanos-concurrency, --thanos-filter-time-start, --thanos-filter-time-end, --thanos-filter-label, --thanos-filter-label-value, and --thanos-aggr-types. See #9262.

• FEATURE: dashboards/alert-statistics: add pending and firing alerts stats; fix query in FIRING over time by group panel. See #10571. Thanks to @sias32 for the contribution.

• FEATURE: vmalert: add random jitter to concurrent periodical flushers targeting the remote write destination. This helps spread remote write flushes across the flush interval, avoiding congestion at the remote write destination and enhancing queue data consumption. See #10729.

• FEATURE: vmalert: expose vmalert_remotewrite_sent_rows and vmalert_remotewrite_sent_bytes histograms to provide better visibility into remote write request sizes. See #10727.

• FEATURE: vmagent: allow setting -1 value for -remoteWrite.maxHourlySeries and -remoteWrite.maxDailySeries command-line flags. This automatically sets limits to the highest possible value in order to enable tracking without enforcing any limits. This is helpful for estimating current usage before applying real limits. See #9614.

• FEATURE: vminsert in VictoriaMetrics cluster: optimize vminsert buffer size per vmstorage node based on available CPU, memory and storage node count to reduce OOM risk. See #10725.

• FEATURE: vmsingle and vmstorage in VictoriaMetrics cluster: allow setting -1 value for -storage.maxHourlySeries and -storage.maxDailySeries command-line flags. This automatically sets limits to the highest possible value in order to enable tracking without enforcing any limits. This is helpful for estimating current usage before applying real limits. See #9614.

• FEATURE: vmalert: expose vmalert_remotewrite_queue_size and vmalert_remotewrite_queue_capacity to facilitate monitoring of remote write queue usage. See #10765.

• FEATURE: vmui: CSV export on the Raw Query tab now includes all labels from the executed query. VMUI no longer prepends a header row, as it is now provided by the backend. See #10667 and #10666. Thanks to @lawrence3699 for the contribution.

• FEATURE: vmsingle and vmselect in VictoriaMetrics cluster: add header row to /api/v1/export/csv output and auto-detect header rows during import via /api/v1/import/csv. See #10666. Thanks to @andriibeee for the contribution.

• FEATURE: all VictoriaMetrics components: expose operating system name and release version as metric vm_os_info. See #10481.

• FEATURE: vmalert: align group evaluation time with the eval_offset option to help manage group execution more effectively. See #10772.

• BUGFIX: vmbackup, vmbackupmanager: retry the requests that failed with unexpected EOF due to unstable network to S3 service. See #10699.

• BUGFIX: All VictoriaMetrics components: Fix an issue where unsupported metric metadata type was exposed for summaries and quantiles if a summary wasn't updated within a certain time window. See metrics#120 and metrics#121.

• BUGFIX: vmagent: fix scrape_series_added metric to update only on successful scrapes, aligning its behavior with Prometheus. See #10653.

• BUGFIX: vmselect in VictoriaMetrics cluster: prevent partial responses from second-level vmselect nodes in multi-level cluster setups. Ensures response completeness and correctness, and avoids cache pollution in top-level vmselect. See #10678.

• BUGFIX: vmsingle and vmstorage in VictoriaMetrics cluster: Fix storage connection saturation spikes at 00:00 UTC and improve data ingestion when the storage is restarted during the first hour of the day. See 10698.

• BUGFIX: vmsingle and vmstorage in VictoriaMetrics cluster: prevent rare panic during storage start-up at 100% disk usage. See #10747 Thanks to @nmn3m for the contribution.

• BUGFIX: vmsingle and vmstorage in VictoriaMetrics cluster: prevent data ingestion from getting completely stuck when storage is under heavy load. See #10784. Thanks to @fxrlv for the contribution.