VictoriaMetrics/VictoriaMetrics v1.1.0-victorialogs

on GitHub

Released at 2024-12-05

• FEATURE: add first and last pipes for returning the first N and the last N logs after sorting them by the given set of log fields. For example, the following query returns up to 5 logs with the biggest value for request_duration over the last hour: _time:1h | last 5 by (request_duration).

• FEATURE: sort pipe: add an ability to apply limit individually per group of logs via partition by (...) syntax. For example, the following query returns up to 3 logs with the smallest request_duration individually per each host: _time:5m | sort by (request_duration) limit 3 partition by (host).

• FEATURE: format pipe: allow formatting log fields in lowercase and uppercase via <uc:field_name> and <lc:field_name> syntax. This can be useful when some fields must be consistently transformed to the same case during querying. See this issue.

• FEATURE: web UI: add frontend-only pagination for table view.

• FEATURE: web UI: improve memory consumption during data processing. This enhancement reduces the overall memory footprint, leading to better performance and stability.

• FEATURE: web UI: reduce memory usage across all tabs for improved performance and stability. See this issue.

• FEATURE: Grafana Loki data ingestion: use Loki stream labels as VictoriaLogs stream fields by default. The set of stream fields can be overriden via _stream_fields query arg or via VL-Stream-Fields header as described here.

• FEATURE: OpenTelemetry data ingestion: use resource labels as VictoriaLogs stream fields by default. The set of stream fields can be overriden via _stream_fields query arg or via VL-Stream-Fields header as described here.

• FEATURE: data ingestion: expose vl_bytes_ingested_total counter at /metrics page. This counter tracks an estimated number of bytes processed when parsing the ingested logs. This counter is exposed individually per every supported data ingestion protocol - the protocol name is exposed in the type label. For example, vl_bytes_ingested_total{type="jsonline"} tracks an estimated number of bytes processed when reading the ingested logs via json line protocol. Thanks to @tenmozes for the idea and the initial implementation.

• FEATURE: data ingestion: expose vl_too_long_lines_skipped_total counter at /metrics page. This counter tracks the number of the ingested lines with the length bigger than the value of -insert.maxLineSizeBytes command-line flag. Such lines are ignored.

• BUGFIX: /select/logsql/stats_query_range API: properly handle limit pipe after sort pipe. Previously the limit was applied globally across all the calculated stats, while it must be applied individually per each step on the start ... end time range. See this issue.

• BUGFIX: vmui: fix for showLegend and alias flags in predefined panels. See this issue

• BUGFIX: fix too big number of columns detected in the block panic when the ingested logs contain more than 2000 fields with different names per every log stream. See this issue for details.

• BUGFIX: properly parse lines after too long JSON lines and Elasticsearch lines with the length exceeding -insert.maxLineSizeBytes. Previously all the lines after the too long line in the stream were ignored.