questdb/questdb 8.3.2

on GitHub

8.3.2 is a stability release, bringing a number bugfixes and improvements. There have been improvements to materialized views, checkpoints, zfs support, and instance metrics, and of course, our built-in web console!

Materialized Views can now be configured to refresh on a timed schedule, instead of immediately whenever new data is written. You can also modify the TTL and refresh limit post-creation, giving you more flexibility to optimise current and new views.

Taking checkpoints will now consume much less disk space, and we've also added some new metrics to help you to monitor your overall database's health, and be alerted to writing delays or suspended tables.

The web console now allows you to set the instance name and colour. This is especially useful for production users, who have multiple deployments and flick between them. This should make it easier keep track of things, and label the instance with a useful name for its users.

This release will shortly be followed by the first beta release of the new ARRAY type. This is a multi-dimensional array (think NumPy), which will initially support DOUBLE values. This will be accompanied by the first upgardes for the ILP clients, to move from the text protocol to a bespoke binary protocol. This means arrays can be sent to the database efficiently, without expansion into lots of text, and is the first step towards supporting all of QuestDB's rich features through the database clients.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Breaking changes 💥

• The greatest and least functions will now only return a null if all of the inputs are null, instead of any. This aligns the behaviour with MSSQL.

• The WITHIN function now defaults to a different implementation, which runs WITHIN before LATEST BY.

  • If you were using the old implementation, which ran LATEST BY first, you can re-enable it using query.within.latest.by.optimisation.enabled=true. This variant was only used for WHERE + LATEST BY + indexed symbols, and at no other time.
  • Or alternatively, use the new implementation and a subquery to force the LATEST BY to run first.

Highlights

UI

• The web console now allows you to set an instance's name and colour! This is handy when switching between different deployments and instances, to make sure you run your queries on the correct console.
• The query log UI element has been upgraded, fixing a few snags, and now supports horizontal scrolling for long rows.
• (Enterprise) SSO users will not have to log back in every time they log out; instead, an existing SSO session can be resumed, or you can switch to a different account.

Prometheus Metrics

New metrics have been added to help monitor the performance of tier-1 (WAL) and tier-2 (Table) storage:

• suspended_tables; this is a global counter for how many tables are currently suspended.
• wal_apply_seq_txn; this is a global counter for sequencerTxn, the txn number corresponding to the latest transaction written to WAL, for a particular table.
• wal_apply_writer_txn; this is a global counter for writerTxn, the txn number corresponding to the latest transaction visible for read, for a particular table.
• You can subtract the above two numbers and track this over time, to estimate if the database is backing up and falling behind on data visibility.
• You can also track replication lag by comparing metrics between the primary and the replica. More granular information can be obtained by polling wal_tables() and sys.telemetry_wal;

Materialized Views

• Materialized Views can now be refreshed on a timer schedule.
  • CREATE MATERIALIZED VIEW price_1h REFRESH START '2025-09-12T00:00:00.000000Z' EVERY 1h AS ...
  • You can also alter existing views to swap to a new scheduled refresh:
  • ALTER MATERIALIZED VIEW price_1h SET REFRESH START '2035-09-12T00:00:00.000000Z' EVERY 1d;
• You can now add a TTL post-creation:
  • ALTER MATERIALIZED VIEW view_name SET TTL 42 DAYS;
• You can now add a refresh limit post-creation, which bounds how far back an incremental refresh will go
  • ALTER MATERIALIZED VIEW view_name SET REFRESH LIMIT 12 HOURS;
• Base table names are no longer case-sensitive.

Checkpointing

• The storage cost to have an active checkpoint has been dramatically reduced. This means you are less likely to run out of storage if you have out-of-order (O3) writes whilst CHECKPOINT CREATE is active.
  • If you encounter any issues, you can roll back to the old version using cairo.txn.scoreboard.format=1.
  • This change also fixes max txn in-flight errors, which could be caused by a leaked query.

Geospatial

• We have upgraded the WITHIN operator to work for generic WHERE clauses, instead of just LATEST BY queries. This should make it much easier to filter for data by geohash location.

Changelist

• fix(sql): fix string constant unescaping in JIT by @jerrinot in #5649
• fix(core): remove critical log message when processing multiple transactions on dedup table by @ideoma in #5650
• fix(core): fix deadlock in logging when handling disk full FS error by @ideoma in #5634
• feat(core): reduce active checkpoint disk cost by @ideoma in #5444
• fix(core): fix startup logging to only check existing dirs for supported file system by @ideoma in #5658
• fix(sql): missing tables and materialized views are not critical errors by @jerrinot in #5654
• feat(sql): ALTER MATERIALIZED VIEW SET REFRESH LIMIT and SET TTL by @puzpuzpuz in #5638
• feat(core): add prometheus monitoring metrics that track WAL apply progress, suspended tables by @mtopolnik in #5660
• fix(wal): reading and writing all wal-e data via mmap to address ZFS mixed IO issues by @amunra in #5656
• fix(sql): non-deterministic function not rejected when creating materialized view by @puzpuzpuz in #5655
• fix(sql): case sensitive base table name when creating materialized view by @puzpuzpuz in #5672
• fix(sql): breaking change 💥- greatest(V) and least(V) now return null only if all args are null by @nwoolmer in #5669
• fix(sql): NullPointerException when calling vararg functions with no arguments by @puzpuzpuz in #5681
• fix(core): adjust query cache scaling to prevent memory exhaustion by @jerrinot in #5683
• fix(core): fixed table metadata commands, incorrectly displayed which column is the designated timestamp by @glasstiger in #5684
• fix(sql): fixed error reported when trying to add an already existing column with IF NOT EXISTS by @glasstiger in #5675
• fix(sql): fix segfault when using within(geohash) with latest on and partition column is empty by @bluestreak01 in #5688
• fix(pgwire): prevent stale results in LATEST ON queries with bind variables by @jerrinot in #5687
• feat(sql): REFRESH MATERIALIZED VIEW RANGE by @puzpuzpuz in #5662
• feat(sql): add multi-datatype support for (LEAD/LAG) window functions by @kafka1991 in #5524
• feat(sql): breaking change 💥 - parallel filtering geohashes using WITHIN by @nwoolmer in #5682
• feat(sql): timer refresh for materialized views by @puzpuzpuz in #5695
• feat(web-console): instance naming UI by @emrberk in questdb/ui#427
• fix(web-console): wait for all body to be streamed before calculating network time by @emrberk in questdb/ui#429
• fix(web-console): jit compiled icon shrink problem and notifications height by @emrberk in questdb/ui#428
• feat(ui): make SSO re-authentication optional on logout by @glasstiger in questdb/ui#412