At a high level, this release contains

• Support for PostgreSQL 15
• Improvements to caching
• Improvements to vacuum engine

Requirements

PostgreSQL: 12.x, 13.x, 14.x, 15.x
TimescaleDB: >= 2.7.0 single-node (multi-node is not supported in this release)
Promscale Database Extension: >=0.8.0

Promscale version compatibility matrix

Upgrade notes

• Before starting the upgrade please make sure that you have deployed version 0.8 of the Promscale extension. You will also need TimescaleDB extension version 2.7.0 or above available. Then stop all Promscale Connector instances, upgrade the first one and restart it. The migration will happen automatically. Once the migration is completed, upgrade the remaining Promscale instances and restart them if needed.

Added

• Add support for PostgreSQL 15
• Alerts from Promscale monitoring mixin are grouped also by namespace label [#1714]
• Added a new family of metrics tracking database maintenance jobs durations and failures [#1745]
• Allow config options in the file to be set as yaml mappings [#1737]
• Add startup.dataset option in the config file for the dataset
  configuration. Supersedes startup.dataset.config which accepts a string
  instead of a mapping [#1737]
• Add an alert to notify about duplicate sample/metric ingestion. [#1688]
• Add histogram to track the number of samples/metadata/span sent per request [#1767]

Changed

• Reduced the verbosity of the logs emitted by the vacuum engine [#1715]
• The vacuum engine now throttles the number of workers used based on the oldest txid from
  the chunks needing freezing [#1761]
• In order to reduce the overall load on the system, some internal database
  metrics won't be collected as often as they used to. None of the affected
  metrics are expected to change faster than its new collection interval [#1793]
• Aggregate metrics at a global level to avoid legend pollution in dashboards [#1800]
• The vacuum engine now looks for compressed chunks missing stats and vacuums these too [#1804]

Fixed

• Fixing the query behind chunks_uncompressed. The new definition should
  change the baseline value [#1794]

Thanks

We'd like to thank the following users for their contribution to this release:

• @mikberg for #1707
• @debovema for #1301

Changelog

• a7a8f7c - Adding a jitter to the "slow metrics" update starting time - Adjusting alerts relying on promscale_sql_database_chunks_metrics_uncompressed_count
• 84b6775 Add DeepCopy method to dataset.Config
• 8173f2f Add SPM feature in docker-compose
• 0b5196c Add alert to notify about duplicate sample/metric ingestion
• 9df822a Add deprecation warning for startup.dataset.config
• b846443 Add heatmap to visualize samples & spans per req
• f78b2de Add histogram to track number samples/metadata/span sent per request
• 0253640 Add Prometheus metrics to the vacuum engine
• 1279d1c Add some details about Jaeger certification
• 85c784e Add support for PG15
• e56849e Add the troubleshooting guide
• 22b5792 Add the troubleshooting guide
• 3b9edc6 Aggregate metrics at global level to avoid legend pollution in dashboards
• 19d8e3e Alert when PG shared_buffers is smaller than open chunks
• e08049e Allow config options in file as yaml mappings
• 6b9a20b Avoid using total suffix for gauge metrics and apply rate for counters
• 73ec0bc Bump series cache default size
• b5a2566 Change log level in vacuum engine
• fee0e1f Collecting job stats for each maintenance job type and signal. - A test has been moved to the extension repository and another test was adjusted to not over-specify the number of jobs in a metrics test. - Two new families of metrics were added to the dashboard - Histograms are now supported in database-reported metrics
• a23059e Fix Docker compose example
• 2abb1d5 Fix formatting changes
• aeb6a5e Fix incorrect histogram quantile in Promscale dashboard
• ed2303c Fix jaeger env vars in docker-compose.yaml
• 8578fe0 Fix series cache growth
• 89bbdaa Fix typo (?) in MaintenanceJobRunningTooLong runbook
• 03c01c4 Fixing Gosec complaints
• e069098 Fixing the query behind chunks_uncompressed metric by making it rely on a function used by the maintenance jobs.
• 08abf8b Further changes based on feedback
• 19dc42d Grow inverted labels cache if needed
• 114f965 Include Jaeger ingest metrics in Promscale dashboard
• 94e68c9 Increase eviction threshold
• 92ad453 Introuducing a new setting allowing to configure custom polling intervals for individual db metric queries. The actual polling still happens on evalInterval, but this setting can by used to make heavier queries run less often.
• 6b96a7d Merge branch 'master' into include-ns-in-alerts
• 8a6b7b8 Merge pull request #1714 from paulfantom/include-ns-in-alerts
• 168e988 Merge pull request #1747 from paulfantom/dashboard-improvements
• 6c160d0 Move test to extension
• 41360b6 Move tracegen to extension repo
• 148f25d Prepare for 0.17.0 release
• 1d7be7c Prepare for the next dev cycle
• 97a6c13 Preparing for 0.8.0 extension release
• 08ad0cc Promscale dashboard: Use rate in Cache hit ratio panels.
• 675ebd0 Reduce cache size in E2E tests b/c of GH runner memory constraints
• 507e376 Remove TestMetricView
• 2ff573e Remove info_view golden file test
• 3349550 Removing 16 bytes alignment for cache elements
• 850151c Removing tests that are failing upon PGX upgrade and are duplicates of tests from promscale_extension.
• b3c030d Rename err label to reason
• 6b45872 Replace deprecated github actions '::set-output' command
• 6a2579e Revert code to match with v2.39.2 Prometheus release.
• 9ce3f73 Since query timeout and cancel use the same error code we need an additional metric label to track what happened. This should also help with having better alerting control.
• 753aed3 Stop counting idle processes in metrics
• b3e1ea9 Tell renovate about golang versions in Dockerfiles
• ac1cf99 Throttle vacuum workers based on chunk age
• 0a01dd2 Track metrics per workload in vacuum engine
• f5255ab Update CODEOWNERS
• c6e66d0 Update PromQL till Sept 30, 2022
• fde61c9 Update architecture to reflect our Jaeger support
• d631066 Update dataset config migration guide
• 4c29693 Update dependency golang to v1.19.3
• d0bd0b1 Update docker-compose.yaml
• dd24506 Update github actions
• ca1ad91 Update goreleaser/goreleaser-action action to v4
• fca2794 Update pkg/vacuum/vacuum.go
• 15c0283 Update pkg/vacuum/vacuum.go
• 93ed639 Vacuum compressed chunks missing stats
• 7081762 docs/mixin reduce flakiness of PromscaleIngestHighDataDuplication by looking at larger window
• c7738f1 docs/mixin/dashboards: fix hidden panels and stop requiring DS variable
• fc92099 docs/mixin: PromscaleStorageHighLatency alert should not include exec method
• e8830ab docs/mixin: add variables to Promscale dashboard
• 84a6bd1 docs/mixin: adjust linter as offending metrics were removed
• ad5118a docs/mixin: adjust minimum step for range queries
• cb580cd docs/mixin: fix exclusion rules
• da6b3c9 docs/mixin: group by namespace
• c3ad7d3 docs/mixin: humanize values in alerts description
• 07fc3b8 docs/mixin: improve Promscale dashboard
• e416b13 docs/mixin: simplify PromscaleStorageHighErrorRate alert
• 840b394 docs/mixin: split query error high alert by handler and use different threshold for query_range
• 16eee06 fix docker-compose
• 4546827 fix docker-compose (fix CI)
• 260db96 refactor: unify promdata proto struct recycle