Highlights
- DuckDB 1.5.6 and Substrait v0.102.0.
- Subquery support in the consumer: uncorrelated subquery expressions can now be read from Substrait plans. Correlated subqueries will follow in a later release.
- Emit mappings are honored on every relation, not only
ProjectRel. Before this, a mapping such as[2, 0]on another relation type could quietly return the wrong columns. - Fewer crashes on malformed or unusual plans. Several plans that used to kill the process with SIGSEGV now fail with a clear error instead. These include an unset
RelorExpression, unrecognized enum values, and enum arguments on aggregates.
Behavior changes
- NOT LIKE is now emitted as
not(like(x, pattern)). It used to be emitted asnot_equal, andnot_equalis now always read as inequality. Plans written by earlier versions of this extension encoded NOT LIKE asnot_equal, so they will now be read as!=. Those plans did not follow the spec, and the old workaround also broke real comparisons:WHERE s != 'a%c'came back asNOT LIKE. (#286) - When an aggregate has more than one grouping set, its grouping set index column is no longer included in the output, so the root names match the output columns. (#291)
- Stricter emit validation. These are now rejected: mappings that are explicitly empty or whose indices are out of range, an emit on a
WriteRel, and an emit on a lateral right-sideFilterRelthat is folded into the join condition. (#274) - Delete plans must filter the table they delete from. A plan that filters one table and names a different table as the delete target is now rejected, because it used to delete rows the plan never selected. (#274)
local_filesreads bind columns tobaseSchemaby name, including nested fields and Hive partition columns. A mismatch withbaseSchemanow raises an error. (#298)
What's Changed
Features
- feat: add subquery support by @andrew-coleman in #290
Fixes
- Fix local files base schema by name by @mbrobbel in #298
- fix: support aggregates whose variant is selected by an enum argument (
std_dev,variance,median) by @andrew-coleman in #294 - fix: honor emit mappings beyond ProjectRel by @alexandrefimov in #274
- fix: do not crash on a plan with an unset Rel or Expression oneof by @andrew-coleman in #296
- fix: empty-table optimizer folding in
get_substraitby @thgschwind in #269 - fix: name an unsupported enum value instead of crashing on it by @alexandrefimov in #295
- fix: emit NOT LIKE as not(like(..)) instead of not_equal by @andrew-coleman in #286
- fix: preserve timestamp literal precision by @alexandrefimov in #275
- fix: leave the grouping set index out of an aggregate's output by @alexandrefimov in #291
- fix: dialect has missing variants for arithmetic aggregates by @andrew-coleman in #284
- fix: correct two wrong DuckDB mappings in duckdb_dialect.yaml (
bitwise_xor,PRECISION_TIMESTAMP_TZ) by @andrew-coleman in #283 - fix: emit integer literals for TINYINT and unsigned integer types by @thgschwind in #280
- fix: map substrait extract(UNIX_TIME) to date_part('epoch') by @thgschwind in #271
- fix: emit TIMESTAMPTZ constants as precision_timestamp_tz literals by @thgschwind in #273
Build
- build: update duckdb to 1.5.6 by @andrew-coleman in #304
- build: update Substrait to v0.102.0 (includes window bound validation and typed RANGE offset fixes) by @bvolpato in #265
Tests & Chores
- test: document upcast-width intent and correct TIMETZ skip comment by @thgschwind in #281
- chore: keep CodeRabbit's summary out of the PR description by @nielspardon in #299
New Contributors
- @alexandrefimov made their first contribution in #275
- @bvolpato made their first contribution in #265
- @mbrobbel made their first contribution in #298
Full Changelog: v1.5.5.1...v1.5.6