timescale/promscale 0.2.0

on GitHub

This release adds support for TimescaleDB 2.0 multinode. This means all of TimescaleDB 2.0 features are now fully supported. This also means that Promscale now supports horizontal scalability across the entire stack!

This release also includes performance improvements and bug fixes.

At a high-level, this release:

• Adds support for Multinode TimescaleDB.
• Improved promQL query latency by 4x in some cases.
• Reduced I/O used by the PostgreSQL stats collector substantially by changing autovacuum settings.
• Fixed metrics produced by Promscale itself
• PromQL engine supports @ modifier which is disabled by default. (see promql-evaluation-flags)
• Added configuration for query timeout and default step interval
• Improved UX

Notes for people upgrading from 0.1.4 and before

• The CLI and ENV option install-timescaledb was renamed to install-extension
• Two new flags are added upgrade-extensions by default set to true will upgrade extensions if newer versions are available and upgrade-prerelease-extensions by default set to false enabling it will upgrade extensions to pre-prelease versions if pre-release versions are available.
• We have changed the namespace of the metrics Promscale itself exposes from ts_prom to promscale. We have also updated the PromQL engine based metrics to have namespace as promscale instead of prometheus. So, metrics like prometheus_engine_query_duration_seconds will now be promscale_engine_query_duration_seconds.
• If running into ERROR: out of shared memory (SQLSTATE 53200) during upgrade, please increase your max_locks_per_transaction setting.

Notes for people upgrading timescaleDB 1.x -> 2.x

• You should run SELECT remove_compression_policy(format('prom_data.%I', table_name), if_exists=>true) FROM _prom_catalog.metric; after the upgrade. This cleans up the old way of running compression jobs, compression will still work (just in the new way).

Prom-Migrator

• Adds support for concurrent pulling and pushing to improve migration throughput. (Please note concurrent push is disabled by default as we've seen some issues migrating data to Thanos concurrently, which we are still working out).

Thanks

@jnadler for PR #456
@jnadler for pointing out that we don't print a warning when an extension is not upgraded due to a schema lock (#431).
@puck for pointing out limitations with password masking for passwords with spaces (#413)
@oleotiger for working with Netdata to make sure Netdata can export data to Promscale
@franck102 for reporting a bug with compression jobs (#419)
@DMOsipov for reporting a bug with reset_metric_retention_period function (#401)

Changelog

557067a Add ARM builds for docker
35cd658 Add alerting, recording rules & data deletion docs
034186f Add compression and uncompression sizes in SCHEMA_INFO.metric.
1a80d18 Add flag to enable optional features in PromQL.
5cbeab3 Add test for PromQL @ modifier
f89da9b Add upgrade extensions flags to upgrade extensions to major releases & pre-releases.
7449bb3 Adds support for concurrent flushing in the writer.
da3f7e6 Adds support for configuring PromQL engine evaluation via CLI flags.
7d0e4ba Adds support for including application_name in db connection string.
231fc4c Adds support to make db-connection timeout configurable via CLI.
6d0838b Adjust autovacuum parameters
becb8fc Change cache size metrics to gauges
8bda75d Consider metric evaluations in query endpoints & update engine namespace.
6bd7f60 Create upgrade test for multinode
7581d04 Fewer logging of non-leaders in debug log-level.
24c7317 Fix docker dependencies for testcontainers
af4f73c Fix error compressing locked metrics
e934247 Fix error handling
3cc5788 Fix logging when extensions need an update but the lock was not obtained.
f56dcc2 Fix optional handling of Promscale extension
d635c63 Fix promQL integration tests
cba66f1 Fix proper handling of install flag
de8f32a Fix the reset_metric_retention_period function
fc8b286 Fix: disable pushing of progress-metric when disabled from cli.
32b3375 Get rid of multinode global & other refactoring
5cf8cfc Improve SQL queries for PromQL
d4d3e19 Improve handling of name clauses
6135b9b Improve logs when connector unable to migrate.
4324e6b Log once per metric table in case of a repetitive duplicate sample.
f529ffb Make upgrade test add second node after upgrade
063d7ae Mask passwords with spaces
53ba48d Prepare for the 0.2.0 release
5f87ccd Prepare for the next development cycle
1e5517b Print promscale version info when provided with version flag.
837065f Refactor dependency management out of migrate code
95135f0 Refactor extension migrations
56cc5fd Refactor runner package.
5a8d25f Small fixes for PR comments
b886512 Support adding nodes in multinode deployments
c9e44b3 Update PromQL upto Jan-20 (@ addition)
2adf969 Update copyright year to 2021
b935926 Update default concurrent-push values in prom-migrator.
4450414 Update docker.md with more details and examples.
a517956 Update findMinMaxTime() from upstream (undo 3eae912).
8c430ca Update prom-migrator docs.
71cf3e4 Update promql dir upto 34656bbf8eb8f45422bf24a9720ea087ab4e6434 which is 7 jan (prom lib update).
2c849e8 Warn about low background workers when less than databases count.
621ec0f add additional checks to parse flags when env variables are set. i.e cfg.InstallExtensions, cfg.HasGroupLockID, cfg.Migrate, & cfg.UseVersionLease.
f20f790 add bare metal promscale stack deployment docs
961b05d add docker-compose file, so that we can refer this compose file in office promscale docs in timescale website.
eaf6576 add tests for docker-compose file
00e266c fix to ignore mock* extension versions
3b546ec increase go tests timeout to 30m
031c020 prom-migrator: adds support for concurrent pulling of data.
44fe18a test CI issue fix
5496570 update new promscale arch image

Docker images

• docker pull timescale/promscale:0.2.0
• docker pull timescale/promscale:0.2
• docker pull timescale/promscale:latest