duckdb 0.9.0

0.9.0 Preview Release "Undulata"

on Python PyPI

This preview release of DuckDB is named "Undulata" after the aptly named Yellow-billed duck native to Africa.

Note: Again, this release introduces a backwards-incompatible change to the on-disk storage format. We suggest you use the EXPORT DATABASE command with the old version followed by IMPORT DATABASE with the new version to migrate your data. See the documentation for details.

What's Changed

• [Dev] Merge master into feature by @Tishj in #7535
• Issue #7563: make_timestamptz by @hawkfish in #7597
• Add support for nested laterals by @arhamchopra in #7528
• Issue #7563: epoch_us(temporal) by @hawkfish in #7629
• Fix lingering clang-tidy issues by @Mytherin in #7670
• Add list_intersect, list_has_any, and list_has_all by @maiadegraaf in #7518
• Issue #7563: epoch_xs(temporal) by @hawkfish in #7648
• Pivot - dynamically switch between using filtered aggregates or the new pivot operator by @Mytherin in #7688
• Add wildcard to JSON Path by @lnkuiper in #7624
• [Dev] Add optional build flag to disable assertions in debug mode by @Tishj in #7618
• [DEV]: ICU C Casts by @hawkfish in #7715
• List_resize by @maiadegraaf in #7401
• Issue #7187: AsOf Join Performance by @hawkfish in #7607
• Some minor CI changes by @samansmink in #7763
• Binder coverage by @hawkfish in #7791
• Vacuum Completely Deleted Row Groups by @Mytherin in #7794
• Issue #7187: AsOf Coverage by @hawkfish in #7774
• Implement FIELD_IDS for parquet writes by @lnkuiper in #7696
• Optimize Regexp_matches to LIKE statements when possible by @Tmonster in #7264
• Jemalloc configuration, more buffer allocator, and remove redundant string copying in parquet dictionary by @lnkuiper in #7697
• Truncate Database File on Checkpoint by @Mytherin in #7824
• LEFT JOIN ON TRUE support by @taniabogatsch in #7821
• Issue #7809: Segment Tree Performance by @hawkfish in #7831
• C Data Interface: duckdb_arrow_scan and duckdb_arrow_array_scan by @angadn in #7570
• Update Julia to 0.8.1 by @Mytherin in #7932
• Add conn.interrupt() to DuckDB python API by @henrinikku in #7895
• renaming part of extension build refactor PR by @samansmink in #7926
• fix swapped x/y regression parameters by @MartinNowak in #7855
• [Docs] Aggregate function README.md by @hawkfish in #7881
• PhysicalPiecewiseMergeJoin improvement by @xuke-hat in #7832
• Initial set of commits to add support for zOS (an IBM mainframe operating system) by @v1gnesh in #7805
• test(nodejs): add test_all_types.test.ts by @Mause in #7740
• Issue #7879: Missing JDBC TIMESTAMP_TZ by @hawkfish in #7922
• Attempt to fix CI on Windows 32 and Python on Windows by @carlopi in #7961
• Fix 7947 by @lnkuiper in #7963
• test: patch test_7652 to skip on pyarrow<11 by @gforsyth in #7966
• NodeJS: Add columns() method to get type info from prepared statement by @Maxxen in #7948
• Fix: Don't free arrow children explicitly by @Maxxen in #7917
• CSV Rejects table by @Maxxen in #7681
• Issue #7809: Segment Tree Performance by @hawkfish in #7891
• Add tpch benchmark run exclusively on parquet files by @Tmonster in #7519
• Bidirectional check storage + minor CI fixes by @carlopi in #7955
• [Swift] fix #7985 by @tcldr in #7993
• Move @samansmink's extension_header_rename.patch by @carlopi in #8001
• [Python] Properly use NumPy array stride when scanning object arrays. by @Tishj in #7964
• CI - No longer run on PR synchronize, instead run on ready_for_review by @Mytherin in #8007
• Parallel pipeline execution should call NextBatch on first batch by @bleskes in #7978
• Micro-optimization for generating collation keys by @Krechals in #7983
• Multiple assignment for UPDATE SET by @nickgerrets in #7968
• CI job to move synchronized PRs to draft by @carlopi in #8010
• [ADBC] ConnectionGetTableSchema and StatementSetSubstraitPlan Functions by @pdet in #7914
• Issue #7852: Window Vectorisation by @hawkfish in #7996
• Moving JDBC Linux x64 builds to CentOS 7 by @hannes in #7991
• CI Draft - token is called GH_TOKEN by @Mytherin in #8016
• Add support for materialized CTEs by @kryonix in #7126
• Reduce memory usage of Parquet writer by @lnkuiper in #7995
• CI auto draft: pass token via environment + avoid wrapping action by @carlopi in #8024
• CI autodraft: use implicit variable [test] by @carlopi in #8027
• remove duplicate pivots declare by @douenergy in #7992
• Fix typo in fts indexing exception by @alexanderchiu in #8034
• Fix issue 7988 by @samansmink in #8023
• Delete DraftMe.yml by @Mytherin in #8048
• Fix 3eb9ab3: Remove unneeded move by @carlopi in #8038
• [CI] Skip many more CI jobs for pull requests, and add make coverage-check to run coverage locally by @Mytherin in #8046
• Extension build configuration refactor by @samansmink in #7735
• Compressed Materialization by @lnkuiper in #7644
• [Relation] Add support for creating an empty ValueRelation by @Tishj in #7967
• Join Order Optimizer has duplicate enumerations and lost some neighbors by @lokax in #7358
• Fix CI wasm by @carlopi in #8057
• [CI] More CI reduction and clean-up by @Mytherin in #8052
• Restore auto-draft functionality by @carlopi in #8058
• Move WebAssembly.yml to NightlyTests.yml by @carlopi in #8060
• Unskip, attach HTTPFS test, and create HTTPState when the opener is not available by @pdet in #8012
• CI fixes: Don't persist ccache for nightlies by @carlopi in #8075
• Fix regression & fix draft mechanism by @carlopi in #8071
• CI compliance feature branch by @carlopi in #8070
• Fix python flaky test (potentially GET requests gets back 403) by @carlopi in #8074
• [Arrow] Fix segfault in appending list data by @Tishj in #8042
• Issue #7852: Window Vectorisation by @hawkfish in #8050
• CONTRIBUTING.md by @carlopi in #8077
• Add ORDER BY clause to query in test_bool.test by @Flogex in #8082
• ART test and benchmark refactor by @taniabogatsch in #8055
• Update plan cost runner script to remove 20% threshold for cardinality estimates by @Tmonster in #7989
• Fix #8067 by @lnkuiper in #8090
• ART prefix refactor by @taniabogatsch in #7930
• Bump Substrait by @pdet in #8110
• Merge feature into master by @Mytherin in #8136
• Increase memory limit in test to prevent non-deterministic CI failures by @lnkuiper in #8138
• UNNEST binder fix by @taniabogatsch in #8111
• Out-of-Core Hash Aggregate by @lnkuiper in #7931
• Add Unittests for ODBC by @maiadegraaf in #7875
• Hive types by @lverdoes in #7674
• CppCheck & CodeQL on master by @carlopi in #8113
• Fix CI for Nodejs and OSX by @carlopi in #8148
• Skip failing test on R on CI by @carlopi in #8161
• Disable failing R tests for now, and disable test_arrow_progress on Windows by @Mytherin in #8155
• Allow disabling of extension loading (through a CMake flag) and allow selectively disabling specific file systems by @Mytherin in #8152
• Fix FTS + ATTACH by @lnkuiper in #8153
• fix: cli parser issue by @stephaniewang526 in #8107
• integrate with julia tables by @aplavin in #7984
• Add JoinReftype to Relational Joins (to add asof, positional, dependent joins) by @Tmonster in #7987
• Adjust the logic for adding type dependencies to temp tables by @jwills in #8073
• Add File filters to Logical Get / Physical Table Scan. by @Tmonster in #7986
• Issue #8119: TO_TIMESTAMP Returns TIMESTAMP by @hawkfish in #8127
• [Python] Start of DuckDB Spark API by @Tishj in #8083
• Create NamingConvention.md, with @samansmink by @carlopi in #8166
• Casts from and to BIT type by @nickgerrets in #7941
• [Python] Add Value class to explicitly set type of prepared parameter by @Tishj in #8114
• Use many linux image in several places by @samansmink in #8097
• Update tests broken after conflicting PRs by @samansmink in #8168
• Auto-generate FormatSerialize/FormatDeserialize code from JSON that defines the schema by @Mytherin in #8156
• Odbc Lambda Removal by @maiadegraaf in #8167
• fix: support case-insensitive lookup in SHOW TABLES by @stephaniewang526 in #8173
• Bump duckdb-wasm, remove patch, fix NightlyTests.yml triggers by @carlopi in #8176
• [Python] Add extra safeguards around join method by @Tishj in #8170
• Bump ubuntu to 18 for linux extensions build by @samansmink in #8179
• user CMAKE_CURRENT_SOURCE_DIR by @peterboncz in #8183
• Auto-generate type serialization + make enum generation deterministic by @Mytherin in #8175
• Fix #7863 - use LoadLibraryW to load extensions by @Mytherin in #8186
• [Python UDF] Disallow create_function as part of a transaction by @Tishj in #8169
• Arrow C API test - initialize result to nullptr by @Mytherin in #8184
• Issue #7969: Prefer Range Join by @hawkfish in #8092
• add verbose option to generate_grammar.py by @douenergy in #8181
• Make the Binder respect the max_expression_depth setting by @Tishj in #8188
• build: upload libduckdb_static.a too by @Mause in #8197
• Pragma platform by @carlopi in #8195
• Issue #7595: AsOf Inequalities by @hawkfish in #8196
• CI Job names to explicit platform they build for by @carlopi in #8202
• ODBC Small Issues fix #7918 #7890 #7653 by @maiadegraaf in #8205
• Enable vcpkg for extensions by @samansmink in #8210
• Remove unnecessary dbplyr fill method by @mgirlich in #8211
• Add union to test_all_types, and arrow and json R/W by @Mause in #7701
• R: Avoid crash when finalizing by @krlmlr in #8207
• Fix #8191 - build duckdb shell error with “DISABLE_UNITY=1” for link … by @zzachimed in #8192
• Chore: Bump duckdb-wasm AND move to emscripten latest by @carlopi in #8220
• Issue #7808: Partition Using PartitionedTupleData by @hawkfish in #8085
• some extensions (like motherduck) take an interest in SET and PRAGMA by @peterboncz in #8226
• Change join ref type to cross if join type is cross and join ref type is regular by @Tmonster in #8227
• Issue #8217: Test Lead/Lag Framing by @hawkfish in #8233
• ODBC: Reformat Diagnosics Function by @maiadegraaf in #8237
• ODBC: Fix for Issue #8190 by @maiadegraaf in #8223
• fix typos and spelling errors in #8244
• (julia) fix show for query result by @tbeason in #8256
• CI: Fix typo in LinuxRelease.yml by @carlopi in #8267
• Remove Node 15 from supported versions by @carlopi in #8251
• Reduce colreaders by @ccfelius in #8248
• Fix init value type in std::accumulate by @xuke-hat in #8275
• Allow parser extensions to use PostgresParser by @ywelsch in #8254
• Implement ROW_GROUP_SIZE_BYTES for Parquet writer by @lnkuiper in #8221
• use cmake_current_source_dir by @samansmink in #8278
• fix(jdbc): result/prepped stmt metadata by @Mause in #8280
• ART: inline row IDs into node pointers by @taniabogatsch in #8112
• Skip row group size test on vsize=2 by @Mytherin in #8292
• Update duckdb-wasm in CI & fix order of operations by @carlopi in #8288
• Issue #8086: Parallel Window Refactoring by @hawkfish in #8269
• [RLE] Emit constant vectors when possible by @Tishj in #8250
• Update R tests to run test_struct.R not on CRAN by @Tmonster in #8297
• Always initialize variables used by ParallelCSVGlobalState::MaxThreads() by @ywelsch in #8295
• [ODBC] Add Row-Wise Fetching and Bool-to-Char Tests by @maiadegraaf in #8306
• Print join_ref_type when explaining joins by @krlmlr in #8276
• Fix handling of cross joins by @krlmlr in #8274
• Remove dangerous overload of cpp11::warning() by @krlmlr in #8272
• add to_base function by @Maxxen in #8122
• Resolve the conflict between the CTE name and the referring table name. by @douenergy in #8302
• Compression for the INT128 type by @nickgerrets in #8277
• Fix some function json defs by @Mause in #8313
• Exposing InterruptState in Sink Combine & Finalize by @samansmink in #8296
• Add iceberg extension by @samansmink in #8317
• Serialization - Add Format(De)Serialize to almost everything by @Mytherin in #8323
• [Python] Add a top-level native python module by @Tishj in #8189
• CI WebAssembly: Avoid eagerly '--recurse-submodules' by @carlopi in #8327
• PR2 for zOS support by @v1gnesh in #7973
• fix(python): restore version field by @Mause in #8338
• Issue 8315: Window Null Order by @hawkfish in #8324
• Swap arguments for atan2 function to atan2(y, x) by @szarnyasg in #8308
• Add Black Python formatter by @Mause in #8335
• [ADBC] Add support for ingestion mode by @pdet in #8349
• feat: python scripts formatting by @Mause in #8354
• [Python] Correctly handle pandas DataFrames when copy_on_write is set to True by @Tishj in #8348
• [Arrow Appender] Clean up file/folder structure by @Tishj in #8289
• [CI] Fix test failure introduced by #8348 by @Tishj in #8361
• Pipeline scheduling issue by @samansmink in #8356
• Workaround for failing vcpkg openssl install by @samansmink in #8362
• Call pwrite/pread in a loop and reduce disk space usage of window_partition_paging.test_slow by @Mytherin in #8369
• Add format serialization for BoundFunction/Aggregate/Window and LogicalGet by @Mytherin in #8365
• Throw exception for encrypted Parquet files by @ccfelius in #8368
• Small out-of-tree extension fixes/features by @samansmink in #8375
• Path separator fix windows by @samansmink in #8345
• Compare scalar functions by name/args/return instead of function object target by @Maxxen in #8364
• [Python] Add support for +/- infinity datetime objects by @Tishj in #8294
• Format serialization - add missing serialization logic for DelimJoin, AsOfJoin, CreateIndex and enums by @Mytherin in #8378
• [CI Python] More 'copy_on_write' fixes by @Tishj in #8376
• fix end of file handling for gzip files by @igorcalabria in #8386
• Issue CI workflows by @lnkuiper in #8390
• [CI] Fix nightly tests (sqllogictests, asof join out of disk space) by @Mytherin in #8385
• use labels field instead of workflow for needs triage by @Mause in #8400
• [Python] Remove PyObject * where possible by @Tishj in #8393
• Fix CMake syntax on non defined variable by @carlopi in #8401
• [RLE] Fix problem created by #8250 by @Tishj in #8391
• [Arrow] Fix conversion from arrow -> duckdb for nested structs by @Tishj in #8383
• [ADBC] Python Tests and Fix for Arrow Schema call. by @pdet in #8366
• Use bot token for issue workflows and increase stale issue budget by @lnkuiper in #8404
• Nitpick: change case of "stale" issue label by @lnkuiper in #8407
• capi: duckdb_interrupt & duckdb_query_progress by @Virgiel in #8064
• Bind the children of a STRUCT alias type. by @Tishj in #8389
• [NanODBC] Fix run_nanodbc_tests.sh by @Tishj in #8388
• ODBC: Add remaining tests by @maiadegraaf in #8382
• fix minor ci failure by @samansmink in #8411
• Update bug_report.yml to ask a better master build question by @Mause in #8427
• Fix capi streaming test - this is not deterministic if there are background threads by @Mytherin in #8429
• bump spatial by @Maxxen in #8394
• read_csv_auto tests: Change 'delimiter' to 'delim' by @szarnyasg in #8409
• [Arrow] Add ChunkScanState interface to preserve chunk-offset when scanning by @Tishj in #8307
• Qualify macros only when called, and throw error on recursive macros by @lnkuiper in #8224
• Add instructions for installing black and clang-format by @Mause in #8373
• Get issue title safely in workflow by @lnkuiper in #8436
• [ADBC] [Python] Adding extra tests and small fixes by @pdet in #8431
• Set total_byte_size of Parquet row groups by @lnkuiper in #8405
• Python TIMESTAMPTZ support by @pdet in #8089
• Add "require noalternativeverify" to all FTS tests by @lnkuiper in #8448
• Check for duplicate member tag names in serialization generation by @Maxxen in #8449
• Issue #5610: Disallow Chained Frames by @hawkfish in #8442
• Issue #8416: Pre-Gregorian Timestamps by @hawkfish in #8418
• Issue #8399: ICU Epoch Offset by @hawkfish in #8415
• Finish specialization of the LIST aggregate function by @taniabogatsch in #8309
• Add Java format config to clang-format by @Mause in #8355
• List_slice adjusted NULL handling and added steps feature by @maiadegraaf in #8087
• DuckDB PySpark Types by @Tishj in #8165
• Fix issue 8230 by @samansmink in #8450
• Move all extensions to new build, remove legacy ci by @samansmink in #8465
• [Dev] Update uncovered files by @Tishj in #8476
• [Python Dev] Use pytest.mark.parametrize to make test logs easier to read in test_all_types.py by @Tishj in #8479
• [C-API + CLI] Add support for named parameters in prepared statements by @Tishj in #7113
• Fix issue 8422: date_part('isodow', timestamptz) to return correct iso day numbers. by @robert-manchester in #8432
• remove static loading extensions for node/python/r dev builds by @samansmink in #8473
• Add geomean macro by @szarnyasg in #8471
• Bump duckdb-wasm to 9f2f87b by @carlopi in #8470
• CI: Change clang-format version to 11 by @maiadegraaf in #8474
• Add list_reverse function by @maiadegraaf in #8477
• R: dbplyr_fill0() no longer exists by @krlmlr in #8480
• General ART improvements and memory pressure reduction by @taniabogatsch in #8437
• [Arrow (Dev)] Refactor arrow scan internals by @Tishj in #8430
• Expose lto and llvm folder compilation flags by @carlopi in #8357
• Documenting R release process by @hannes in #8081
• Remove assertion on empty linked list by @taniabogatsch in #8502
• Fix total_cardinality is zero will result in undefined-behavior by @zhouliqi in #8506
• Issue #8512: Negative Time Intervals by @hawkfish in #8515
• Rework Metadata Storage by @Mytherin in #8513
• Issue #8086: Window Work Stealing by @hawkfish in #8491
• Add android ifdefs by @Mause in #8526
• Fuzzer #590: MAKE_TIMESTAMPTZ Extreme Paranoia by @hawkfish in #8514
• R: Update duckplyr tests by @krlmlr in #8482
• R: Add test for disconnect behavior by @krlmlr in #8483
• [ADBC] Addings more tests and small fixes. by @pdet in #8478
• Refactoring the Join Order Optimizer by @Tmonster in #8468
• [Dev] Fix CI failure caused by #7113 by @Tishj in #8533
• fix query hang in Jetbrains programs (IntelliJ, DataGrip, etc) by @Mause in #8511
• Add PendingExecutionResult::ALL_TASKS_BLOCKED by @samansmink in #8497
• [R] Fix retention of time component of TIMESTAMPS when converting time zones #8547 by @ateucher in #8548
• Add "field ID's" to new serialization by @Maxxen in #8536
• Fix #3170: Get old value when update on rows with big row_id by @zippond in #8520
• Silence warnings on thread sanitizer for CanHave[No]Null by @carlopi in #8552
• Issue #6027: Explicit ROW_GROUP_SIZE no longer breaks IMPORT/EXPORT round-trip by @jmeulemans in #8559
• fix: remove unused ForwardRef import by @Mause in #8543
• [Python] Enable use of CUBE / ROLLUP in groups parameter of DuckDBPyRelation aggregate methods. by @Tishj in #8510
• Adding explicit errors for cascade/set null/set default by @hannes in #8572
• Throw exception on type error in JsonDeserializer by @Maxxen in #8291
• Remove packaging for Python 3.6 by @carlopi in #8577
• Fix gcc weird behaviour around template deduction (#8533) by @carlopi in #8576
• Fix: Remove undefined behaviour in frame-of-reference bitpacking by @nickgerrets in #8569
• Issue #8461: Null Hive Typing by @hawkfish in #8540
• [Python] Don't clog the tools/pythonpkg/duckdb folder on build by @Tishj in #8534
• Add Mold Linker Support for Linux by @douenergy in #8517
• Fuzzer #572: Lead Lag Overflow by @hawkfish in #8538
• Fuzzer #668: Window List NULLs by @hawkfish in #8557
• Update R readme to add instructions on how to load the httpfs extension by @Tmonster in #8567
• [ADBC] Finish ADBC spec compliance by @Tishj in #8417
• fix(#8412): forbid topn optimization when there are scalar subquery in limit or offset clause by @Reminiscent in #8519
• R: Update duckplyr tests by @krlmlr in #8608
• R: Allow microsecond precision for timestamps by @krlmlr in #8597
• R: Fix core dump when failing to evaluate ALTREP query by @krlmlr in #8600
• Fuzzer #583: Orders Nary Aggregates by @hawkfish in #8598
• Add register functions for AggregateFunction used by extensions by @ttanay in #8607
• R: Full support of lists and structs in R by @krlmlr in #8503
• R: Avoid leaking format specifiers in cpp11::stop() calls by @krlmlr in #8614
• [R] Add ADBC integration with the adbcdrivermanager package by @paleolimbot in #8172
• fix(#8441): some filter expressions can not be pushed down by @Reminiscent in #8532
• format.py: add checks (and suggestion) to install needed tools by @carlopi in #8629
• R: Fix warning on mismatched integer comparison by @carlopi in #8626
• Rename branch from master to main by @hannes in #8639
• Second round of branch renames by @hannes in #8643
• Fix: Unimplemented type for TryAddOperator by @nickgerrets in #8566
• typo by @hannes in #8644
• Wrap ALTREP methods with BEGIN_CPP11 and END_CPP11 by @krlmlr in #8647
• Fix test_csv_httpfs.test_slow by @carlopi in #8637
• [Python] Add support for datetime64 columns with a time-unit that is not ns by @Tishj in #8620
• PR3 for zOS by @v1gnesh in #8625
• [Dev] Add explicit expected error for test/sql/storage/multiple_clients_checkpoint_pending_updates.test by @Tishj in #8654
• Extension sha256: Avoid full copies by updating sha context by @carlopi in #8636
• Expose window functions in the Python API by @jarandaf in #8568
• Issue #2827: Julian Day Parts by @hawkfish in #8523
• Issue #8659: Non-Invertible Casts by @hawkfish in #8661
• Fix CI failure with test/sql/copy/s3/glob_s3_paging.test_slow by @samansmink in #8670
• ADBC: Use new(std::nothrow) + check on null by @carlopi in #8676
• Issue #8674: Interrupt Work Stealing by @hawkfish in #8679
• Add fix for substrait CI failure by @pdet in #8545
• fix cpp11 issue in r api by @hannes in #8664
• [Dev] Fix sporadically failing test window-rows-overflow.test by @Tishj in #8678
• [Dev] Use our parquet writer in test_filter_pushdown_2145 test instead by @Tishj in #8694
• fix: typo in comment by @SkyFan2002 in #8696
• CI: skip test/fuzzer/sqlsmith/window-rows-overflow.test by @carlopi in #8693
• Makefile extension config by @carlopi in #8692
• Remove deprecated experimental_parallel_csv configuration option by @szarnyasg in #8669
• [Dev] Fix CI failure related to pandas and timezones by @Tishj in #8665
• Fix for #3789, separate updates into storage backends individually by @hannes in #8618
• [Internals] Fix key/value name in MAP produced by map_entries by @Tishj in #8460
• C-API: Fix duckdb_arrow_scan API by @k-anshul in #8653
• [PythonDev] Fix failing test_statement_bind.py test by @Tishj in #8711
• Issue #3409: Time TZ Storage by @hawkfish in #8650
• [Parquet] Fall back to VARCHAR for unsupported types in EXPORT DATABASE by @Tishj in #8710
• [Parquet reader] support reading small decimals with DBP encoding by @samansmink in #8715
• When projecting to the R client, use GetName instead of ToString() by @Tmonster in #8717
• Custom indexes: Step 0 by @Maxxen in #8367
• Add reusable workflows for out-of-tree extensions by @samansmink in #8714
• feat(jdbc): read struct, map, and unions by @Mause in #8686
• Slight refactor to silence a gcc13 warning by @hannes in #8727
• Change parquet_write_memory_limit.test_slow to not preserve order by @lnkuiper in #8746
• [Python] Pandas 2.1.0 update by @Tishj in #8738
• bump codecov by @Maxxen in #8751
• Order Describe table with column id by @pdet in #8729
• CSV Read: move buffer initialisation to thread local by @pdet in #8672
• Distinct/order modifier column deduplication by @lnkuiper in #8748
• Autoloading mechanism for extensions by @samansmink in #8732
• duckspark, implement createOrReplaceTempView by @hannes in #8759
• Extension load tests: fix-up b014b69 by @carlopi in #8775
• Apply patches and bump extensions by @carlopi in #8765
• Fuzzer #936: Sort Key Exceptions by @hawkfish in #8773
• CSV Sniffer - State Machine by @pdet in #8253
• [Dev] Untangle some really obfuscated code in ReorderTableEntries by @Tishj in #8707
• Use manylinux_2_24 image for Linux aarch64 Python builds by @Mause in #8770
• Radix Partitioned Hash Table Rework by @lnkuiper in #8475
• Issue #7809: Parallel Partition Sorting by @hawkfish in #8593
• Add the ability to access file_path in FileOpener by @stephaniewang526 in #8484
• CAPI: Make it possible to create struct types by @alnkesq in #8455
• Serialization pt 3: mega follow-up by @Maxxen in #8752
• Remove patches and bump dependecies (aws & iceberg) by @carlopi in #8777
• [Python] DuckDB -> Pandas | NULL -> str becomes None instead of 'NaN' by @Tishj in #8745
• [PythonDev] Add timeout to test_multithread.py by @Tishj in #8747
• Bump uncovered_files.csv by @carlopi in #8790
• Avoid rounding down buffer block sizes to uint32_t by @lnkuiper in #8785
• Duckdb wasm loadable fixes by @carlopi in #8766
• [Dev] Handle SKIP_EXTENSIONS as early return in duckdb_extension_load by @carlopi in #8781
• Fix R ci by @Tmonster in #8783
• Switching to twine token and remove cleanup job by @hannes in #8792
• Moving R client out of main source tree by @hannes in #8793
• Re-add pypi-cleanup script and use pypi-cleanup package for deleting dev packages by @Mytherin in #8795
• Fix #8797: Fix off-by-one in varchar -> list conversion code causing out-of-bounds access by @Mytherin in #8799
• Add "ends_with" as an alias of "suffix" by @holdenmatt in #8807
• [Dev] Pass explicitly USE_MERGED_VCPKG_MANIFEST by @carlopi in #8806
• #7412 add sha256 scalar function by @jdnvn in #8723
• Extension docs by @carlopi in #8784
• Fix Python Release Clean-up: need to install requests module by @Mytherin in #8811
• Very minor fixes of CI problems by @carlopi in #8809
• Fix "Too many open files" issues by @lnkuiper in #8812
• Add list_cosine_similarity, list_distance, list_inner_product by @Maxxen in #8750
• Towards stable storage of indexes and the ART by @taniabogatsch in #8703
• JSON bugfixes by @lnkuiper in #8810
• Issue template: Ask contributors to avoid using screenshots in issue reports by @szarnyasg in #8829
• [PythonDev] Remove version lock on adbc_driver_manager by @Tishj in #8725
• Internal #215: Serial Unordered Scans by @hawkfish in #8824
• [Generated Column] Return stringified generated expression in duckdb_columns by @Tishj in #8835
• Extension API: Allow autoloading of extensions dependencies & use it for duckdb_aws by @carlopi in #8839
• Remove references to duckdb_query_graph by @szarnyasg in #8826
• DuckDB-Wasm fixes related to (auto)loadable extensions by @carlopi in #8827
• Revert "Use manylinux_2_24 image for Linux aarch64 Python builds" by @Mause in #8849
• fix(nodejs): thread safety between node instances in single process by @Mause in #8862
• fix: remove TODO's in Node docs by @Mause in #8861
• Switch over to new Serialization Framework by @Mytherin in #8863
• Fixes to error messages: An error occurred ... by @carlopi in #8855
• Attempt at fixing OSX Extensions Release nightly build failures by @carlopi in #8847
• Internal #164: Single Partition Sorting by @hawkfish in #8791
• [Python] Add Expression class by @Tishj in #8469
• Partial Blocks for Index Serialization by @taniabogatsch in #8825
• Update create_package.py for new extension defines by @Mause in #8870
• Wasm extension signing by @carlopi in #8881
• DuckDB-Wasm workflow: Fix syntax by @carlopi in #8883
• Expose SeekPosition in HTTPFS by @Maxxen in #8876
• DuckDB-Wasm workflow: fixes by @carlopi in #8885
• DuckDB-Wasm workflow: Pass on token and secret by @carlopi in #8886
• Use lock when accessing the http metadata cache by @pdet in #8882
• DuckDB-Wasm workflow: Uncomment line by @carlopi in #8887
• Rework Storage of Deletions - Allow for lazy loading and incremental re-writing of deletions by @Mytherin in #8869
• Fix for #8440 by @pdet in #8879
• Issue #8316: Time Cast Statistics by @hawkfish in #8888
• Sqlite commands fixes by @Tmonster in #8848
• Fuzzer #838: Julian DATE Overflow by @hawkfish in #8677
• Remove support for NodeJS 10 by @Mause in #8872
• DuckDB-Wasm workflow: Add credentials explicitly by @carlopi in #8889
• fix some nodejs tests in some envs by @jraymakers in #8774
• Fix link in extension/README.md by @isaacbrodsky in #8904
• [Fix] Verification that CSV File exceed line sizes. by @pdet in #8892
• Add R CMD Check workflow back for duckdb src PRs by @Tmonster in #8845
• DuckDB-Wasm workflow: moving to official repo by @carlopi in #8918
• Fix #8895: Reading .csv.zst files depends on parquet extension (check… by @carlopi in #8906
• ART fixes - no more empty buffers and correctly serializing parsed expressions by @taniabogatsch in #8912
• fix(jdbc): uniform exception handling by @Mause in #8685
• Wrap failing shell-test so it is not tested on windows. by @Tmonster in #8923
• Fix R CMD BUILD script. by @Tmonster in #8921
• Point to R package repo for new issues by @krlmlr in #8919
• [Dev] Only emit ConstantVectors when scan_vector is invoked by @Tishj in #8916
• [Dev] Fix signing scripts so they can be called from arbitrary locations by @carlopi in #8935
• Detect Header columns with nullpadding by @pdet in #8915
• [Parquet | ExportDatabase] Deal with unsupported parquet types in EXPORT DATABASE. by @Tishj in #8798
• Issue: #8932 Fix description in empty_needle_removal.hpp by @Light-City in #8933
• Fuzzer fixes by @Tmonster in #8920
• Add ordering to to underspecified tests by @lnkuiper in #8819
• Add rowcount property for Python cursor objects by @hannes in #8911
• Various fixes for very large (overflow) strings by @Mytherin in #8943
• [PythonDev] Protect test_replacement_scan.py from side effects by @Tishj in #8953
• remove redundant code by @gitccl in #8946
• fix(nodejs): wait to write all data to disk before calling back by @Mause in #8843
• [Python Arrow] Fix issue related to TIMESTAMP_TZ and filter pushdown into PyArrow by @Tishj in #8856
• Add PRAGMA metadata_info that displays info about the storage of metadata, and correctly write free metadata blocks by @Mytherin in #8956
• DuckDB-Wasm: Custom repository to be served over https by @carlopi in #8960
• [PythonDev] Merge pyduckdb module into duckdb by @Tishj in #8959
• [Python] Extend show with keyword arguments to set BoxRenderer options by @Tishj in #8954
• Bump aws (removing patches) & spatial to latest commits by @carlopi in #8936
• [Python] Fix issue with dtype parameter in the read_csv method. by @Tishj in #8387
• Update logo by @szarnyasg in #8970
• [Python] Add 'names' option to read_csv by @Tishj in #8967
• Allow JSON reader to sample across multiple files by @lnkuiper in #8891
• Fix typos and update variable type info in Contributing.md by @thfmn in #8975
• Fix outdated C API docs by @ocadaruma in #8973
• CSV Parallel Tests CI by @pdet in #8910
• Truncate the db file before the WAL for differential storage by @Y-- in #8980
• Fix #8592: correctly deal with torn writes by ignoring SerializationException during initial read by @Mytherin in #8982
• Fixes for CSV And Aggregate Nightly Build Issues by @pdet in #8852
• Internal #317: Fix foreach Label by @hawkfish in #8987
• httpfs: Avoid corruptions with servers sending more data than asked for by @carlopi in #8940
• Move arrow, postgres_scanner and sqlite_scanner to be autoloadable by @carlopi in #8966
• Fix some fuzzer issues by @lnkuiper in #8871
• ART fixes by @taniabogatsch in #8995
• [ODBC] Error Management Improvement by @maiadegraaf in #8939
• fix(nodejs): cpp exception handling by @Mause in #8994
• [PythonDev][Docs] Add missing docs for Expression methods. by @Tishj in #8991
• [Python][DuckSpark] Add Column and a bunch of DataFrame methods by @Tishj in #8990
• [JSON] Support escaping quotes in VARCHAR -> LIST cast by @Tishj in #8655
• Rework row matching by @lnkuiper in #8979
• Issue #7581: Epoch Returns Doubles by @hawkfish in #8963