VictoriaMetrics/VictoriaMetrics v1.91.0 on GitHub

Released at 2023-05-18

SECURITY: upgrade Go builder from Go1.20.3 to Go1.20.4. See the list of issues addressed in Go1.20.4.
SECURITY: serve /robots.txt content to disallow indexing of the exposed instances by search engines. See this issue for details.
FEATURE: update docker compose environment to V2 in respect to V1 deprecation notice from June 2023. See Migrate to Compose V2.
FEATURE: deprecate -bigMergeConcurrency command-line flag, since improper configuration for this flag frequently led to uncontrolled growth of unmerged parts, which, in turn, could lead to queries slowdown and increased CPU usage. The concurrency for background merges can be controlled via -smallMergeConcurrency command-line flag, though it isn't recommended to change this flag in general case.
FEATURE: do not execute the incoming request if it has been canceled by the client before the execution start. See this pull request.
FEATURE: support time formats with timezones. For example, 2024-01-02+02:00 means January 2, 2024 at +02:00 time zone. See these docs.
FEATURE: expose process_* metrics at /metrics page of all the VictoriaMetrics components under Windows OS. See this pull request.
FEATURE: reduce the amounts of unimportant INFO logging during VictoriaMetrics startup / shutdown. This should improve visibility for potentially important logs.
FEATURE: upgrade base docker image (alpine) from 3.17.3 to 3.18.0. See alpine 3.18.0 release notes.
FEATURE: VictoriaMetrics cluster: do not pullute logs with cannot read hello: cannot read message with size 11: EOF messages at vmstorage during TCP health checks performed by Consul or other services. See this issue.
FEATURE: vmagent: support the ability to filter consul_sd_configs targets in more optimal way via new filter option. See this feature request.
FEATURE: vmagent: add support for consulagent_sd_configs. See this feature request.
FEATURE: vmagent: emit a warning if too small value is passed to -remoteWrite.maxDiskUsagePerURL command-line flag. See this issue.
FEATURE: vmalert: add support of recursive globs for -rule and -rule.templates command-line flags by using ** in the glob pattern. See this issue.
FEATURE: vmalert: add ability to specify custom per-group HTTP headers sent to the configured notifiers. See this issue. Thanks to @Haleygo for the pull request.
FEATURE: vmalert: detect alerting rules which don't match any series. See these docs and this feature request.
FEATURE: vmalert: support loading rules via HTTP URL. See this issue. Thanks to @Haleygo for the pull request.
FEATURE: vmalert: add buttons for filtering groups/rules with errors or with no-match warning in web UI for page /groups. See this issue.
FEATURE: vmalert: do not retry remote-write requests for responses with 4XX status codes. This aligns with Prometheus remote write specification. Thanks to @MichaHoffmann for the pull request.
FEATURE: vmauth: add ability to filter incoming requests by IP. See these docs and this feature request.
FEATURE: vmauth: add ability to proxy requests to the specified backends for unauthorized users. See this feature request.
FEATURE: vmauth: add ability to specify default route for unmatched requests. See this feature request.
FEATURE: vmauth: retry POST requests on the remaining backends if the currently selected backend isn't reachable. See this issue.
FEATURE: vmui: add ability to compare the data for the previous day with the data for the current day at Cardinality Explorer. See this feature request.
FEATURE: vmui: display histograms as heatmaps in Metrics explorer. See this feature request.
FEATURE: vmui: add WITH template playground. See this feature request.
FEATURE: vmui: add ability to debug relabeling. See this feature request.
FEATURE: vmui: add an ability to copy and execute queries listed at top queries page. Also make more human readable the query duration column. See this feature request and this pull request.
FEATURE: vmui: increase default font size for better readability.
FEATURE: vmui: cardinality explorer: return back a table with labels containing the highest number of unique label values. See issue.
FEATURE: vmui: add notification icon for queries that do not match any time series. A warning icon appears next to the query field when the executed query does not match any time series. See this feature request.
FEATURE: vmbackup: add -s3StorageClass command-line flag for setting the storage class for AWS S3 backups. See this issue. Thanks to @justcompile for the pull request.
FEATURE: vmbackup: store backup creation and completion time in backup_complete.ignore file of backup contents. This allows determining the exact timestamp when the backup was created and completed.
FEATURE: vmbackupmanager: add created_at field to the output of /api/v1/backups API and vmbackupmanager backup list command. See this doc for data format details.
FEATURE: vmbackupmanager: add commands for locking/unlocking backups against deletion by retention policy. See this doc for data format details.
FEATURE: vmctl: add support for different time formats for --vm-native-filter-time-start and --vm-native-filter-time-end command-line flags. See this issue.
FEATURE: vmctl: set default value for --vm-native-step-interval command-line flag to month. This enables time-based chunking of data based on monthly step value when using native migration mode. See this issue.
BUGFIX: reduce the probability of sudden increase in the number of small parts on systems with small number of CPU cores.
BUGFIX: reduce the possibility of increased CPU usage when data with timestamps older than one hour is ingested into VictoriaMetrics. This reduces spikes for the graph sum(rate(vm_slow_per_day_index_inserts_total)). See this pull request.
BUGFIX: fix possible infinite loop during indexdb rotation when -retentionTimezoneOffset command-line flag is set and the local timezone is not UTC. See this issue. Thanks to @faceair for the fix.
BUGFIX: do not panic at Windows during snapshot deletion. Instead, delete the snapshot on the next restart. See this comment for details.
BUGFIX: change the max allowed value for -memory.allowedPercent from 100 to 200. See this issue.
BUGFIX: properly limit the number of OpenTSDB HTTP concurrent requests specified via -maxConcurrentInserts command-line flag. See this issue. Thanks to @zouxiang1993 for the fix.
BUGFIX: do not ignore trailing empty field in CSV lines when importing data in CSV format. See this issue.
BUGFIX: disallow " chars when parsing Prometheus label names, since they aren't allowed by Prometheus text exposition format. Previously this could result in silent incorrect parsing of incorrect Prometheus labels such as foo{"bar"="baz"} or {foo:"bar",baz="aaa"}. See this issue.
BUGFIX: VictoriaMetrics cluster: prevent from possible panic when the number of vmstorage nodes increases when automatic vmstorage discovery is enabled.
BUGFIX: MetricsQL: fix a panic when the duration in the query contains uppercase M suffix. Such a suffix isn't allowed to use in durations, since it clashes with a million suffix, e.g. it isn't clear whether rate(metric[5M]) means rate over 5 minutes, 5 months or 5 million seconds. See this and this issues.
BUGFIX: vmagent: properly handle the vm_promscrape_config_last_reload_successful metric after config reload. See this issue.
BUGFIX: vmagent: add __meta_kubernetes_endpoints_name label for all ports discovered from endpoint. Previously, ports not matched by Service did not have this label. See this issue for details. Thanks to @thunderbird86 for discovering and fixing the issue.
BUGFIX: vmalert: retry failed read request on the closed connection one more time. This improves rules execution reliability when connection between vmalert and datasource closes unexpectedly.
BUGFIX: vmalert: properly display an error when using query function for templating value of -external.alert.source flag. See this issue.
BUGFIX: vmalert: properly return empty slices instead of nil for /api/v1/rules and /api/v1/alerts API handlers. See this issue.
BUGFIX: vmauth: do not return invalid auth credentials in http response by default, since it may be logged by client. See this issue.
BUGFIX: vmui: fix the display of the tenant selector. See this issue.
BUGFIX: vmui: fix UI freeze when the query returns non-histogram series alongside histogram series.
BUGFIX: vmui: fix the text display on buttons in Safari 16.4.
BUGFIX: alerts-health: update threshold for TooHighMemoryUsage alert from 90% to 80%, since 90% is too high for production environments.
BUGFIX: vmbackup: fix compatibility with Windows OS. See this issue.
BUGFIX: vmctl: fix performance issue when migrating data from VictoriaMetrics according to these docs. Add the ability to speed up the data migration via --vm-native-disable-retries command-line flag. See this issue.
BUGFIX: stream aggregation: fix bug with duplicated labels during stream aggregation via single-node VictoriaMetrics. See this issue.

See full changelog.

How to run VictoriaMetrics

Unpack the victoria-metrics-*.tar.gz archive and read these docs.

vmutils-*.tar.gz archive contains the following tools:

vmutils-*-enterprise.tar.gz archive contains the following additional enterprise tools:

The corresponding docker images are available here.
Cluster version is available here.