github questdb/questdb 8.3.0
on GitHub

latest releases: 9.4.3, 9.4.2, 9.4.1...
14 months ago

8.3.0 moves Materialized Views into an advanced beta. Thanks to all of you have tried the beta and helped us iron out the remaining creases!

This release also brings significant ingestion performance improvements, particularly for small transactions and for batched PG Wire inserts. Not to mention a variety of bug fixes and safety enhancements to improve overall user experience.

But let's not forget about querying data! Check out how your ASOF JOIN queries are doing! We have released dramatic performance improvements for ASOF JOINs that include interval scans and/or post-join filters. Benching against larger datasets has shown 100-1000x speedups for some queries.

For any questions or feedback, please join us on Slack or on Discourse!

See also our prettier release notes page!

Breaking changes 💥

  • Materialized Views will now be enabled by default. If you do not wish to use them, you can disable them by setting cairo.mat.view.enabled=false.

  • Designated timestamp columns can no longer hold values beyond the Year 9999. We found that these values were generally produced by mistaken arithmetic. This prevents undroppable partition issues, and also reduces the chances of a configured TTL dropping your data.

  • Some IPv4 built-in functions were removed. You may need to introduce explicit casts in SQL queries that relied on implicit conversions between ipv4 and other database types. This removes some ambiguity that was causing issues with queries unrelated to ipv4.

  • Converting char to numeric values now aligns with PostgreSQL behaviour, whether implicitly or explicitly casted.

  • SAMPLE BY will now correctly handle a variety of DST-related timestamp grouping issues. SAMPLE BY queries that use DST shifts are more likely to be executeed in parallel, too. These changes mean your result set might look different to before - but should be a consistent, rational output.

  • FLOAT and DOUBLE types, when output to JSON or web console, will now have different precision. Previously, this was limited artificially. Now, it will print as many digits as it can. If you rely on the old output for any integration testing, you may need to upgrade your tests!

Highlights

  • Materialized Views are now in an advanced BETA.
    • Views will be correctly invalidated when the base table is dropped.
    • Syntax errors in materialized view definitions should report the correct error position.
    • PARTITION BY and parentheses () are now optional.
    • Full mat view refreshes should no longer trigger occassional OOM errors.
    • Still to come: a mat view write-performance enhancement, which is also a pre-step to supporting time-range DELETEs!
  • ASOF JOIN queries that include filters over large datasets will now run 100-1000x faster than before!
  • SYMBOL columns can now have their capacity changed in-place, using ALTER TABLE table ALTER COLUMN column SYMBOL CAPACITY 4096!
  • Ingestion of small batches has been sped by up to 100x!
  • Ingestion via PG Wire batched inserts is faster and more memory efficient.

Changelist

  • feat(core): speed up WAL small transaction apply 100x by @ideoma in #5347
  • fix(core): mat view is not invalidated when base table is dropped by @glasstiger in #5486
  • fix(core): make partition drop atomic by @ideoma in #5515
  • fix(pgwire): fix handling of sparse bind variables by @bluestreak01 in #5514
  • fix(core): prevent inserting designated timestamp beyond year 9999, this would create invalid partition name by @mtopolnik in #5478
  • fix(sql): support casting and aliasing timestamp column in materialized view by @eugenels in #5485
  • fix(pgwire): possible unhandled error (double-close) in case of INSERT failure by @bluestreak01 in #5523
  • fix(sql): crash when GROUP BY key includes timestamp by @jerrinot in #5527
  • perf(sql): speed-up of simple projections by @jerrinot in #5529
  • feat(core): accept EntraID's cookie expiry timestamp format by @glasstiger in #5470
  • feat(sql): change column type SQL to support changing symbol column capacity by @ideoma in #5497
  • chore(core): merge identical mem.getDirectVarcharA/B by @mtopolnik in #5542
  • fix(ilp): disable Nagle's algorithm in HttpClient (set TCP_NODELAY flag) by @mtopolnik in #5550
  • fix(sql): stuck thread on high cardinality group by query timeout by @puzpuzpuz in #5539
  • fix(http): unhandled NullPointerException on SQL query error by @puzpuzpuz in #5548
  • fix(core): fix issue with missing WAL segment directory during replication by @eugenels in #5538
  • fix(sql): make the function's argument overload rules consistent with functions' implementation by @kafka1991 in #5528
  • fix(core): fix critical error message in writing data with deduplication and a possible crash on writing same timestamp lines by @ideoma in #5547
  • fix(sql): improve error handling for malformed SQL queries by @kafka1991 in #5572
  • fix(sql): breaking change 💥 - remove some ipv4 operator functions. by @kafka1991 in #5566
  • fix(core): fix table suspended on concurrent ALTER and RENAME operations by @ideoma in #5563
  • fix(core): fix cannot invoke "io.questdb.network.Epoll.close()" error by @eugenels in #5573
  • perf(sql): breaking change 💥 - parallel SAMPLE BY with time zone execution by @puzpuzpuz in #5493
  • fix(sql): accept long256 values via hex-encoded string in ILP by @nwoolmer in #5579
  • fix(http): breaking change 💥 - fix incorrect FLOAT type scale in JSON response by @bluestreak01 in #5578
  • fix(sql): spurious cancelled by user errors returned by queries executed in parallel by @puzpuzpuz in #5582
  • perf(pgwire): improve performance of batch inserts via PostgreSQL driver by @bluestreak01 in #5564
  • chore(core): ensure mat view logging cannot stall by @bluestreak01 in #5587
  • perf(sql): expanded fast-path support for non-keyed ASOF JOINs by @jerrinot in #5553
  • chore(sql): enable materialized views by default by @puzpuzpuz in #5460
  • fix(sql): fix show create table union query issue. by @kafka1991 in #5577
  • fix(ui): show JSON parsing error in case of an invalid query result by @emrberk in questdb/ui#411
  • fix(ui): fix incorrect unit normalisation for wal row throughput by @emrberk in questdb/ui#414
  • fix(ui): don't use navigator.clipboard in insecure context by @emrberk in questdb/ui#417
  • fix(ui): workaround for broken safari copy schema mechanism by @emberk in questdb/ui#418
  • feat(ui): add storage details, persistent expand states, symbol details by @emberk in questdb/ui#409

New Contributors

Full Changelog: 8.2.3...8.3.0

Check out latest releases or
releases around questdb/questdb 8.3.0

Don't miss a new questdb release

NewReleases is sending notifications on new releases.

Get notifications