questdb/questdb 6.3 on GitHub

Release highlights

The JIT compiler is now enabled by default. The compiler aims at significantly improving execution times for queries with simple arithmetic expressions in where statements. This feature can be disabled by setting cairo.sql.jit.mode to off.
The where clause is executed concurrently, which include both JIT-based and function framework based filters. Additionally SQL optimiser takes into account limit values to optimise data passes and optimise memory usage
Database backup via filesystem snapshot support has been added for cloud environments, e.g. AWS EBS. This enables live database backups to be inexpensive both in terms of effect on ingestion and SQL execution performance as well as cloud cost. Snapshotting is a three prong process. First, cloud administrators issue statement SNAPSHOT PREPARE, which locks all table readers and flushes filesystem caches to disk (through the sync syscall). Second, administrators can at this point invoke the snapshotting procedure, of their cloud vendor, which can be left to run for however long it needs. Third, statement SNAPSHOT COMPLETE is called to release the readers and should be issued as soon as possible even as the snapshot is being produced. This feature can be disabled by setting cairo.snapshot.recovery.enabled to false.
Configuration settings validation is now strict. Unsupported settings and typos cause startup failure (logs are produced with appropriate instructions on how to migrate).
Add new functionalities in the web console: SQL formatting and autocomplete table names.

Gotchas

If you upgrade 6.2.1 -> 6.3 you cannot go back to 6.2.1 without restoring from backup.

What is new

Storage engine

Enable JIT compiler by default #2090.
Add asynchronous API for chaining SQL execution components and expose concurrent where clause execution results #1732.
Add filesystem snapshotting support for cloud environments. Includes statements SNAPSHOT PREPARE|COMPLETE #1907.

SQL

Add cast function to allow casts from string to boolean, e.g. cast('true' as boolean) #2041.
Add date_trunc() function to truncate timestamps, e.g.: date_trunc('week', '2022-05-07T02:09:30.111111Z') -> '2022-05-02 00:00:00.0' #2017.
Add flush_query_cache() function to clear all Http and PGWire worker caches explicitly by user request #2028.
Optimise latest by on symbol column with no index, preventing in many cases a full table scan #2010.
Make is null an alias for = null and is not null for != null for null comparisons. This is required to enable integration with 3rd parties, specifically MindsDB #1967.
Hide telemetry table from tables() SQL output. This behaviour is governed by setting telemetry.hide.tables #1951.
Add sum() function for type long256 #1982.
Add comparison operators for type long256 #1943.

ILP

Allow users to change the default column type (done through config settings e.g. line.integer.default.column.type) when ILP adds a new column automatically #2040.

PGWire

Add config setting pg.security.readonly to toggle read-only mode #2021.

HTTP

Use status code 400 to indicate errors. Prior to this change, status 200 would be returned regardless #2039.
Add geohash support for CSV import and export #1971.

Web Console

Add SQL formatting feature #1999.
Add link to Slack community in web console #1962.
Support query and executeQuery URL params for inserting and executing queries in web console #1927.

Miscellanea

Index rebuild, string col index recover utils #1978.
Made config settings validation optional, but enabled for new installs #2047.

What we fixed

Storage engine

Always test the existence of a table by checking the filesystem #2046.
Fix metadata validation to avoid occasional errors while adding new column #2054.
Fix reader reload after truncate table #2022.
Handle buffer overflow in async command queue #2008.
Handle Table writer commit error in ILP #1989.
Fix alter table queue overflow error #1959.
Follow-up fix for o3 commit lag shift #1934.
Load text_loader.json as a file when it cannot be found as a resource #1929.
Writer supports notification callbacks when table is extended #1961.
Solve a race condition that happened in ILP #1908.

SQL

Jit to handle bind sql context change #2053.
Fix timestamp designated timestamp column data type validation #2038.
Ignore null literals for non-nullable types #2000.
Fix brace matching inside cast #2036.
Fix ignored where clause (order by hour) #2027.
Fix designated timestamp filter with now() function applied incorrectly in some cases #2014.
Parsing ALIGN TO CALENDAR; with semicolon at the end #2013.
Fixed set operations/distinct and column pushdown #1956.
Limit implicit string, symbol -> timestamp cast to constants to fix expressions like symbol = int #1977.

ILP

Log parsing error on disconnect when disconnect on error is enabled #2085.
Tolerate space symbol at the line end #1940.
Add default values for ILP props #1945.
Race between schema inference and schema mutation could corrupt data #2081.

PGWire

Rudimentary support for COPY FROM via pgwire #2061.
Fix persistent function parser issue #1997.
Passing boolean as 1 and 0 bytes over binary wire protocol #1941.

HTTP

Fix memory leak in REST API #1983.

Web Console

Bump minimist from 1.2.5 to 1.2.6 #1988.
Bump node-forge from 1.2.1 to 1.3.1 #2003.
Add appendAt option to appendQuery #2024.
Another appendQuery erroneous case #1994.
QueryPicker to support nested structure #1992.
Adjust regex to allow brackets in editor tokens #1985.
Fix unresponsive ui on hidden tables #1963.
Error highlighting on selections #1965.
Fix copy to clipboard for safari #1960.
Fixed conditional update of Web Console from RT package #1957.
Add help button in web console #1952.
Remove redundant console.log #1932.

Miscellanea

Fix trailing digits in double conversions #297, #2055.
Added timeout rule to help troubleshoot ci test freezes #2044.
Fix flaky query cache flush tests #2035.
Remove outdated comment #2033.
Add two tests to exemplify use of java.sql.Timestamp with micro precision #1995.
Validate server Properties #1998.
Stress o3 commit #1990.
Assert ts column size #1987.
Fixed logging and status reporting for windows service mode #1969.
Handing corrupt _txn file in Mig620 #1933.
Fixed PG ODBC driver compatibility problems #2031.
Fixing code coverage in build #2100.

Documentation

Updated URLs to Web Console docs #2049.
Updated try questdb section in readme #2051.
Added instructions to build with the web console #2032.
Updated CMake build instructions #1922.
Remove section about commit squashing #1924.
Fix contributor badge #2084.

Contributor's Hall of Fame

We want to thank our new contributors: