timescale/promscale 0.11.0 on GitHub

This release is a major step forward in our evolution of Promscale. It makes distributed tracing generally available and moves a lot of our logic into the Promscale database extension, allowing for further data optimization. Upgrades from previous versions of the system come with more caveats, please see below before upgrading.

Please note this release should only be used with the single-node version of TimescaleDB.

At a high level, this release

General Availability of OpenTelemetry trace data support. This includes a fully customizable, out-of-the-box, and modern application performance management (APM) experience in Grafana using SQL queries on OpenTelemetry traces, to get new insights immediately after deployment.
Native support for PromQL-based alerting and recording rules so you centralize management of those instead of doing it individually in each Prometheus instance and also so you can use the Prometheus agent instead of full Prometheus.
Promscale extension is mandatory starting with this release, as all the SQL codebase has been migrated to the Promscale extension. This has been done to improve performance and the query experience.
Native support for the Jaeger query API in Promscale. Now you can connect a Jaeger data source in Grafana directly to Promscale without the need for the Jaeger-query component.
Resource recommendation guide for traces to help you allocate the required compute and storage resources for your needs.
Out-of-the-box alerts, Dashboards, and Runbooks to monitor Promscale and help you configure your production environment.

Upgrade notes:

Previous tracing data will be dropped when you upgrade. In prior releases, tracing was only supported in beta. To improve performance we had to make some fundamental structural changes to the way data was stored. Unfortunately, these changes require dropping data.
Note that this release contains quite a few changes relative to the previous version. Thus, we suggest testing the upgrade and taking a backup if possible.
Before starting the upgrade please make sure that you have deployed version 0.5.0 of the Promscale extension. You will also need TimescaleDB extension version 2.6.1 or above available. Then stop all Promscale Connector instances, upgrade the first one and restart it. The migration will happen automatically. Once the migration is completed, upgrade the remaining Promscale instances and restart them if needed.
This upgrade could take up to 5-10 minutes on very large databases. During that time Promscale will not be able to ingest or query data.
This upgrade modifies many database objects. Thus on large deployments, you may need to increase the max_locks_per_transaction parameter on your database before starting the upgrade.
The only HTTP verb /healthz endpoint supports is GET. Some load balancers (e.g. HAProxy) that relied on the old behaviour need to be re-configured to use the verb GET instead.

Requirements

PostgreSQL: 12.x, 13.x, 14.x
TimescaleDB: >= 2.6.1 single-node (multi-node is not supported in this release)
Promscale Database Extension: >=0.5.0

Deprecations

Legacy HA implementation i.e. based on pg_advisory_lock has been removed.
Removed deprecated flags.

Additions

Configuration options to set up global parameters like compression and retention that were previously only configurable via SQL. This is to help better support the deployment of Promscale via GitOps.
Add database SQL stats to the Prometheus metrics exposed through the /metrics endpoint for improved monitoring of Promscale. These metrics are available under the promscale_sql namespace.

Improvements

Tracing in Promscale is now by default enabled and you shouldn't need --enable-feature=tracing and tracing.otlp.server-address=:9202 as the OTLP gRPC server by default listens to port 9202.
Performance improvements for querying traces by altering the indexes.
Rename tracing enums to user-friendly names on the SQL side, changed them to lowercase, and removed the prefix.
Metric tables and views are now owned by prom_admin.

Fixes

Disable the option to run Promscale both in high-availability and read-only mode.
Register promscale_ingest_channel_len_bucket metric and make it a gauge [#1177]
Log warning when failing to write response to remote read requests [#1180]
Fix Promscale running even when some component may fail to start [#1217]
Fix promscale_ingest_max_sent_timestamp_milliseconds metric for tracing [#1270]
Fix prom_api.reset_metric_retention_period on two-step continuous aggregates [#1294]

Helm Chart Improvements

Automatic configuration of Promscale alerts.
Ability to set Promscale dataset configuration before installation.
Remove option to enable tracing in Promscale connector, as now by default the Promscale connector is enabled with tracing support.

Changelog

892cf88 *: add docs formatter and checker
9eaf677 *: format documentation and fix broken links
7daa364 Fix docs formatting
f49197a Prepare for the 0.11.0 release
c3c95d6 Revert "Use ha image in Docker compose file"
5b8727e Switch docs to using the ha docker image
ae56c16 Update deprecated entries in promscale.conf
55c6480 Use ha image in Docker compose file
4e42aa7 *: update data types to otel 0.49.0
df2aee4 .github: add @onprem as code owner for deployment methods and monitoring mixin
d3d8f94 Fix output of skipped extension prerelease information
c7acfc8 Fix tests
8489d7b Log mandatory requirement of Promscale extension when upgrading
0c32fe2 Make a unique temp table for each table type
2554e1c Moving temp table creation into the extension to be able to suppress DDL logging.
ce20e46 Prepare 0.11.0-alpha.3
33a7443 Publish packages for ubuntu distros
29f82da Replace deprecated type imports with recommended ones.
e99e03d Reuse temp table for trace ingest
3f7aa10 Update -metrics.rules.prometheus-config -> -metrics.rules.config-file
542db6d Update dependency otel/opentelemetry-collector to v0.50.0
efdff99 Update golang dependencies
9a3e1ef missed one
339c69b pkg/internal/testhelpers: use new format for defining mounts
e58c9f5 Fix lint issue
e7f1a6a Optimize span query based on maximum time duration
4c333f7 Prepare the 0.11.0-alpha.2 release
5c4a2d1 Rework jaegerquery wiring to pass down config
3638e21 Update github actions
b9eae0d Update github actions
892f272 Add runbooks in docs.
d2d8932 Prepare for the 0.11.0-alpha.1 release
f5d9c89 deploy/helm-chart: properly set dataset configuration and provide a way to set the config file for promscale
5e25565 deploy: regenerate
a287a61 docker-compose: add a flag to allow promscale start and improve test resilience
cb07f26 *: add testing alert syntax in CI
5df37a7 *: reconfigure renovate-bot
70161d8 .github: add dashboard linter to CI
d7abe5d .github: control tooling version used in CI via renovate
2836fb1 .github: finer control over used goreleaser version
892f57c .github: fix setting labels on PRs by renovate bot
b12b97c .github: move checking for alerts propagation to helm workflow
89e91d0 .github: move renovate config to decluter TLD
2d9f392 .github: remove dependabot
f90d8d2 .github: switch code ownership to a subteam
400ae0c Add CLI flags and support for reading Prometheus config.
c8970a6 Add Gitops dataset configuration options
b12fa94 Add Jaeger query HTTP server.
e37886e Add JamesGuthrie as owner of pgmodel/querier
36e8907 Add ability to stop routines on partial error.
2700b27 Add alerting rules for Promscale.
231904a Add alerts for database check metrics.
69109d3 Add alerts for database sql metrics.
9842067 Add compression job performance hotfix instructions
e7eae31 Add database metrics into pgxconn package.
6a005c4 Add lifecycle docs on ingest-adapter appender.
ee8a8ec Add metrics for health-check.
d8583e3 Add missing word to slow ingestion warning
91b4e34 Add note saying prom-migrator cannot migrate to HA setups.
9e7afaf Add proper CICD docker image resolution in upgrade_tests.
a539a20 Add renovate.json
43b0e26 Add support for testing database metrics module.
526a660 Add support to expose database metrics in Promscale.
e91daaa Add test case for reset_metric_retention_period with two-step agg
bdcc960 Add workflow dispatch for e2e tests
2be878e Additional improvements and adding input parameters section to select data sources for Postgres and Jaeger
a973379 Adds support for /api/v1/rules API
f563ae4 Adjust to handle new default search path
ac605ef Better debug logs on failed tests
01fc4c1 Bump github.com/open-telemetry/opentelemetry-collector-contrib/pkg/translator/jaeger
5b2c63c Bump go.opentelemetry.io/collector/model from 0.44.0 to 0.45.0
d2f39ee Bump go.opentelemetry.io/otel/exporters/jaeger from 1.3.0 to 1.4.1
3f5c573 Bump minimum TimescaleDB version to 2.6.1
53a1898 Bump otel deps
02e6a1b Bump securego/gosec from 2.9.6 to 2.10.0
c963c8e Change end-to-end tests to work on latest db
477829e Change extension schema to "public"
9e79cb3 Codeowners update
301d9f6 Delete .travis.yml
bfa90a1 Deprecate testing for TimescaleDB 1.x
0fa7486 Disable running Promscale in both HA and read-only mode
a7e71e7 Enable tracing flags by default
c7fc7e9 Expose build information through Prometheus metrics
75c2ac4 Filter on jsonb hash so planner can use index
0ff2efa First iteration of APM experience on Grafana
0b7437e Fix Docker image typo in end-to-end tests
d7aa13e Fix promscale_ingest_max_sent_timestamp value for traces.
8642fa5 Fix batch IsFull check
b652d76 Fix broken link in README
7b233db Fix documentation around add_prom_node
8e6a8f7 Fix golden file tests
4247167 Fix ingested_samples telemetry by using other name for received insertables.
af67201 Fix issues related to relying on default dashboards and improve tagging
4c64156 Fix test to use pre-release extension
71acec0 Fix testcontainers breaking change
da3d44a Fixes for new tag_map type
d49da56 Fixes to testing in response to PR comments
87dca5c Handle pre-release tags in CICD
60550fa Implement /api/v1/alerts API.
816c9f9 Implement adapters for queryable and appendable.
30b2139 Implement gorilla mux router to unify Prometheus APIs and Jaeger APIs.
bf1a706 Implement managers related to ruling/alerting.
d3fc7be Improve logging of active servers on start.
923881b Improve performance of common errors query by querying the span view instead of the event view
ff3cf4b Improve query performance of node graph queries and update queries used in variables to ensure they only those options for which there is data to be shown
47f6769 Improve upgrade test output
8d88c5b Ingest traces using COPY FROM
42f2752 Link with runner and implement manager.Stop()
65fbad1 Log warning when remote read request is cancelled midway
3f31aca Make Renovate bot ignore pinned Prometheus version
7c62c32 Make prom_admin owner of metric tables and views
58da7b2 Make the extension required and delegate migrations to the extension
21c992c Minor cleanups of dataset documentation
ea478b9 Now "the oldest" image is the one on pg12.
1c8efec Prepare for the 0.11.0-alpha release
da8c0a9 Prepare for the next development cycle
e49e74b Provide example for configuration via config.yml
eaac09e Re-enabling PG12 tests.
72d4962 Reduce dependabot to update Go deps weekly
65a6891 Refactor PromQL-engine creation to be in client.
3d7cd20 Refactor rule-manager implementation to consider noop.
2a7ce3b Refactor telemetry metric registration.
0070f1f Refactor testing setup without TimescaleDB extension
8e871f0 Register promscale_ingest_channel_len metric and make it a Gauge.
6e0f9bb Remove CHECK constraints on trace tables
342f61c Remove Multinode upgrade test until we fix the extension
e6ef77b Remove StartTime and EndTime from QueryHints
f714452 Remove deprecated flags for new release
60c6e39 Remove documentation for old HA setup
bcdafad Remove legacy HA implementation
3a230d9 Remove testing for TimescaleDB-OSS
f1a3644 Rename Throughput and Latency to Requests and Duration. Fix other inconsistencies. Keep variables when changing between dashboards
9ec2ee7 Rename duration_ms to Duration. Clean up unsused config
7b85613 Rename tracing enums
48d003d Require Promscale Extension in Tests
df8041a Run tests against corresponding extension docker container
f100e73 Simplify validation of Prometheus config and starting of rules manager.
e319f76 Support pre-release versions of 0.5.0
5b2206b Temporarily disable pg12 tests
6f58451 Track number of ingested bytes for metrics and traces
a0cae7d Transfer and use local mod files for migration-tool.
6528ab1 Update E2E test timeout to 30 min
257dd94 Update E2E tests to use internal Jaeger wrapper.
4848c4c Update dependency golang to v1.17.7
b02157e Update dependency golang to v1.18.1
9a70a6a Update dependency otel/opentelemetry-collector to v0.46.0
38fdad0 Update docker-compose
1c592ce Update docker-compose
4cd7a91 Update docs/mixin/dashboards/apm-home.json
f6f10c1 Update docs/mixin/dashboards/apm-home.json
bf7a4c3 Update docs/mixin/dashboards/apm-service-overview.json
1cd4dcc Update github actions
7836383 Update github actions
dd6fea0 Update github actions
c49026f Update golang
256668e Update golang dependencies
8e90b2e Update golang dependencies
14a7774 Update golang to v0.46.0
e617740 Update golangci/golangci-lint-action action to v3
dfeab2f Update logo and add link to docs page
609311e Upgrade PGX to latest master
498cafd Use getter/setters for default values
20b6b13 Use order by column index for consistency
f4e40d3 Use trace_id custom type
5064999 Verify telemetry engine nil if TimescaleDB is not installed.
4edc91c deploy/helm-chart: add ability to set dataset config parameters
b9a5dd8 deploy/helm-chart: add automatic provisioning of alerts via helm chart
79536c1 deploy/helm-chart: conditionally set password and user in connection secret
e254561 deploy/helm-chart: remove openTelemetry.enabled option
4bc5181 deploy: regenerate
0d30f11 deploy: regenerate
5c00671 docs/mixin: add basic promscale dashboard
ba7d8f1 docs/mixin: add runbook_url link
0a87098 docs/mixin: jsonnetify mixin to allow using common mixin tooling
f98afe3 fix links to hotfix /rollback files
8aaf5c5 goreleaser: Fix deprecated 'use_buildx'
5484848 move goreleaser config to cannonical location
a5fc5fd renovate: Do not show version of dependency being update for gomod