paradedb/paradedb v0.27.0-rc.0 on GitHub

What's Changed

fix: use AggregateScan when PostgreSQL drops a GROUP BY key by @mdashti in #6608
fix: run AggregateScan when PostgreSQL moves a HAVING condition to WHERE by @mdashti in #6638
ci: increase RHEL publish runner disk space by @philippemnoel in #6655
chore: Generate Dockerfiles for v0.26.0 by @paradedb-github-bot[bot] in #6645
ci: follow Docker Scout v1 and configure Docker Hub authentication by @philippemnoel in #6657
ci: document action version pin constraints by @philippemnoel in #6660
ci: fix upgrade coverage and consolidate PostgreSQL setup by @mdashti in #6647
perf: keep the first range partition's NULLs with an exclusion, not a union by @mdashti in #6592
perf: pin Tantivy at the Exclude probe that does not walk excluded runs by @mdashti in #6658
perf: Visibility checking cleanups by @stuhood in #6526
perf: Defer fetching ctids to the top of join plans by @stuhood in #6665
fix: Ensure column access is injected in all plan shapes by @stuhood in #6675
fix: honor interrupts while advancing heap filters by @rebasedming in #6670
fix: respect RLS leaky-qual ordering in filter pushdown by @aryanpatel-ctrl in #6614
feat: Add ability to compute topk in joinscan as an aggregate by @barbarj in #6494
fix: Basescan was dropping quals under certain conditions by @barbarj in #6682
ci: Add Wikipedia benchmarker runs by @rebasedming in #6689
perf: Fix plans for range-partitioned three table disjunction by @stuhood in #6669
fix: Solve a generic plan's parameters into their own ExprContext by @mithuncy in #6692
fix: invalidate typmod caches on subtransaction abort by @mithuncy in #6566
perf: Raise default IVF leaf size to 200 by @RuchirRaj in #6681
fix: detect the search operator before PostgreSQL simplifies it away by @mdashti in #6424
test: accept native index scan plans under concurrent writes by @philippemnoel in #6709
docs: deprecate the key_field index option by @rebasedming in #6303
ci: Upload query benchmark results before publishing by @philippemnoel in #6714
feat: Add support for pdb.agg() OVER () to join scan by @barbarj in #6684
perf: Use Top K parallel worker planning for aggregates by @rebasedming in #6688
fix: Skip per_tuple_cost penalty for search operators in the target list by @sainad2222 in #6461
ci: Compact query data when squashing gh-pages history by @philippemnoel in #6715
chore: update Docker runtime and benchmark dependencies by @philippemnoel in #6661
fix: publish parallel Base Scan worker telemetry at shutdown by @aryanpatel-ctrl in #6540
fix: bound NUMERIC scale when routing a field to Numeric64 by @IamYipi in #6387
fix: highlight through the aliased field a query matched by @devdattatalele in #6271
fix: index and search date values of infinity and -infinity by @wisper-dev in #6727
chore: update Rust and developer tooling versions by @philippemnoel in #6662
feat: Include parallelism in aggregate EXPLAIN ANALYZE output by @rebasedming in #6720
perf: Use statistics for simple query estimates by @rebasedming in #6701
chore: update Rust dependencies and Nix snapshots by @philippemnoel in #6663
docs: enforce snippet coverage and validate ORM setup by @philippemnoel in #6716
fix: install ICU headers for source image builds by @philippemnoel in #6741
perf: Count query matches directly on fully visible segments by @rebasedming in #6737
fix: joinscan now declines pdb.agg() specs with repeated keys on the same path by @barbarj in #6744
fix: install ICU headers in publish builds by @philippemnoel in #6755
ci: run StackExchange and Wikipedia with benchmarker by @jamessewell in #6758
fix: commit background merges locally and install PostgreSQL's SIGTERM handler by @philippemnoel in #6708
feat: Convert existing sql window functions to be computed in the same aggregate step as topk by @barbarj in #6502
feat: take the read's SIREAD lock in the ParadeDB scans by @mdashti in #6392
chore: Generate Dockerfiles for v0.26.1 by @paradedb-github-bot[bot] in #6766
ci: refresh docs snippet coverage digests after the 0.26.1 sync by @barbarj in #6768
chore: remove Docker Scout policy checks by @philippemnoel in #6772
perf: Wire optional posting bitmaps for faster counts by @rebasedming in #6691
ci: Report p95 and QPS for benchmarker workloads by @rebasedming in #6784
perf: Discount match-all and term-only count traversal by @rebasedming in #6779
ci: ignore the release version in docs snippet coverage digests by @barbarj in #6771
ci: count #[pg_test] coverage in the system unit-test job by @wangkhc in #6781
chore: Switch join_aggregate_window_facet to pdb.agg. by @stuhood in #6805
fix: Evaluate STABLE search predicates at execution time by @philippemnoel in #6798
fix: debug-build panic in NOT phrase queries by @mithuncy in #6794
fix: install ICU headers for Antithesis builds by @philippemnoel in #6809
fix: route pdb.query rhs through per-operator dispatch in |||, &&&, ### by @devdattatalele in #5851
chore: repin Tantivy to the #317 squash commit on main by @mithuncy in #6811
chore: bump rustls to 0.23.46 for the cargo-audit failure by @mithuncy in #6815
perf: run heap filters only on the index matches in AggregateScan and JoinScan by @mdashti in #6773
feat: support window functions with parameterized limits over join queries by @barbarj in #6778
fix: keep range filter score on fully covered segments by @mithuncy in #6775
chore: refresh two regress snapshots left stale by the #6773 merge by @mdashti in #6821
perf(vector): enable the exact plan and Auto located path for vector scans by @RuchirRaj in #6769
refactor(vector)!: remove the graph router and adopt tantivy's vector setup by @RuchirRaj in #6802
feat: break down vector buffer hits and include parallel workers by @rebasedming in #6827
fix: search predicates above an Append opened the partitioned parent index by @devdattatalele in #6325
docs: preserve signatures in enterprise release syncs by @philippemnoel in #6764
fix: release a bitmap intersection stream when its cursor drops by @devdattatalele in #6449
fix: decline JoinScan pushdown for cross-scale NUMERIC join keys (#6100) by @DeviousCardi in #6352
fix: parallel bitmap intersection crash when its Gather is rescanned by @devdattatalele in #6751
feat: Make mutable segments append-only by @rebasedming in #6788
fix: pin Mintlify CLI to a published version for docs CI by @rebasedming in #6826
fix: prevent double-conversion of range partition bounds for NUMERIC fields (#6104) by @abelmarnk in #6373
feat: cover indexable disjunctions with BitmapOr by @devdattatalele in #6418
fix: preserve vector parallelism for runtime filters by @rebasedming in #6835
perf: avoid unnecessary parallel reader setup by @rebasedming in #6824
fix: merge small segments when every background layer exceeds the target segment size by @wangkhc in #6761
feat: Support for combining GROUP BY with pdb.agg in window functions by @stuhood in #6806
chore: Split join_aggregate_topk_count by selectivity by @stuhood in #6847
chore: Do not require SQL upgrade script renames by @stuhood in #6849
fix: omit latency results for unmet recall targets by @rebasedming in #6846
feat: Support aggregate filters (e.g. COUNT (*) FILTER (WHERE ..) by @barbarj in #6819
fix: accumulate parallel worker buffer and WAL usage under MPP by @devcool20 in #6789
fix: launch the background merger under backpressure and release its pins per candidate by @mithuncy in #6855
perf: Ensure that InList dynamic filters are applied by @stuhood in #6848
chore: Fixes for the 0.26.2 release. by @stuhood in #6865
fix: Centralize pg->arrow type mappings by @barbarj in #6767
chore: Generate Dockerfiles for v0.26.2 by @paradedb-github-bot[bot] in #6864
perf: size default segments by query parallelism and partition keys by @rebasedming in #6874
feat: stream IVF training by bumping tantivy to TrainingSource by @walter-woodall in #6795
fix: Improve handling of empty relations by @stuhood in #6888
fix: handle range bounds at and beyond the edges of integer fields by @wisper-dev in #6757
fix: unicode_words only removes emojis when remove_emojis is set by @wankhede04 in #6873
test: wait for other snapshots before the VACUUM in join_hot_update_vacuum by @mdashti in #6894
test: accommodate worker skew in parallel build test assertion (#6648) by @devcool20 in #6892
fix: Re-enable joinscan support for post-datafusion postgres projections by @barbarj in #6803
fix: reject out-of-range date values during index build and insert (#6730) by @devcool20 in #6876
chore: update Tantivy to latest main by @rebasedming in #6916
fix: remove 1K document flush cap for vectors by @rebasedming in #6901
fix: binary-search benchmark recall targets by @rebasedming in #6918
perf: push every ORDER BY key into the join aggregate TopK by @mdashti in #6921
refactor: use pg_sys constants for built-in function OIDs by @barbarj in #6913
chore: bump rustls-webpki and webpki-roots to fix RUSTSEC advisories by @barbarj in #6936
fix: Improve column naming for whole-row references and non-columnar system columns in joinscan decline messages by @barbarj in #6917
fix: restore legacy range tokenizers after upgrade by @rebasedming in #6940
fix: increase Debian, Ubuntu, and schema publish storage to 80GB by @philippemnoel in #6943
fix: act on cancel and SIGTERM during DataFusion scans, regex walks and IVF training by @mdashti in #6886
chore: Add explanations of missing symbol errors. by @stuhood in #6890
fix: register true variant of unicode_words_removeemojis tokenizer as fallback if remove_emojis is false by @barbarj in #6952
chore: Split disjunctive join query by selectivity by @stuhood in #6919
fix: cancel the MPP worker's request wait when the leader is signalled by @mdashti in #6934
fix: Normal scan panics with min > max when pushed-down LIMIT exceeds 8192 by @barbarj in #6959
fix: Preserve cancellation during VACUUM mutable flush by @philippemnoel in #6957
fix: array terms facets over a join under MPP by @mithuncy in #6949
fix: reclaim temporary memory during heap filter evaluation by @devcool20 in #6782
chore: pin Tantivy to buffered-union seek_danger fix by @walter-woodall in #6958
feat: Add joinscan support for window function FILTERs by @barbarj in #6935
feat: Allow joinscan to postgres-project volatile expressions by @barbarj in #6965
fix: preserve Numeric64 comparison semantics without overflowing by @kumarUjjawal in #6978
fix: supply a snapshot for logical replication typmod lookups by @philippemnoel in #6982
docs: update ORM versions and 0.26 examples by @philippemnoel in #6984
perf: Warn for a stale visibility map. by @stuhood in #6973
fix: count ParadeDB index scans in pg_stat_user_indexes by @mdashti in #6914
perf: order an all-inner join by the planner's row estimates by @mdashti in #6925
perf: Prune segments before estimation and aggregate execution by @stuhood in #6974
chore: Generate Dockerfiles for v0.26.3 by @paradedb-github-bot[bot] in #6993
chore: extract regression test oid redaction into a shared helper by @barbarj in #6968
chore: Bump the ctutils version to address RUSTSEC-2026-0385. by @stuhood in #7004
fix: credit original PR authors in release notes by @philippemnoel in #7000
ci: move SchemaBot into the repository with pg_query by @philippemnoel in #6719
fix: gate comparison pushdown on collation semantics by @devdattatalele in #5871
fix: Make order by warning clearer by @jamessewell in #6946
fix: construct operator function expressions with makeFuncExpr by @Yanghanrui666 in #6895
ci: standardize RunsOn setup before cache and artifact operations by @philippemnoel in #7010
fix: construct operator function expressions with makeFuncExpr and add regression coverage by @philippemnoel in #7009
fix: make merges and VACUUM's merger wait interruptible by @mithuncy in #6740
test: use a unique tiebreaker in generated_paging_small by @abelmarnk in #7014
test: use a unique tiebreaker in generated_subquery by @philippemnoel in #7019
fix: parse numeric fields in query strings by @kumarUjjawal in #6209
perf: optimised to use doc_count when count aggregate key matches with GROUP BY key by @ygxio in #6707
ci: choose upgrade versions from checked-out migrations by @philippemnoel in #7008
refactor: Consolidate benchmark publishing, snapshots, and workload image builds by @philippemnoel in #6749
feat(pg_search): deprecate in-index ltree support in favor of GiST bitmap intersection (#6093) by @Sunny-200 in #6877
fix: normalize unfielded numeric query literals by @kumarUjjawal in #7025
docs: explain index cleanup for autovacuum by @rebasedming in #7033
fix: apply backpressure across immutable merge layers by @rebasedming in #6880
fix: Check paradedb.term and paradedb.term_set keys for null by @barbarj in #7034
ci: serialize benchmark publication to gh-pages by @philippemnoel in #6994
docs: fix redirects and index the latest release notes by @philippemnoel in #7042
fix: make optional bundled extensions opt-in by @jamessewell in #7044
chore: remove period from Just use Postgres tagline by @philippemnoel in #7047
ci: migrate Cohere vector benchmarks to pdbench and Benchmarker by @rebasedming in #7040
ci: shorten Cohere k6 measurement runs by @rebasedming in #7049
fix: return system columns from JoinScan heap tuples by @BilalAtique in #6887
chore: update datafusion-distributed version by @barbarj in #7059
perf: stage index build segments in PostgreSQL temp files by @rebasedming in #6948
perf: pin Tantivy tip and retire geometry APS early-stop by @walter-woodall in #7020
chore: Remove postgres_early_topk.sql. by @stuhood in #7063
fix: stop NOT (field @@@ NULL) from matching every row by @pavanmanishd in #6998
ci: restore Cohere index build metrics in baselines and comments by @rebasedming in #7061
fix: prevent readers from starving VACUUM cleanup by @rebasedming in #6879
fix(tokenizers): avoid quadratic Lindera token consumption by @puppylpg in #7066
test: stabilize immutable backpressure merge fixture by @mithuncy in #7071
fix: Track qgen recovery coverage per fault episode by @philippemnoel in #6955
chore: Break out a lazy sorted scan. by @stuhood in #7003
perf: Implement rank joins by @stuhood in #6898
fix: allow vacuum delete assertion to remain unreachable by @philippemnoel in #7078
fix: rewrite grouping Vars in mixed Tantivy AggregateScan projections by @aryanpatel-ctrl in #6234
fix: debug-build panics in filtered Top K queries by @mithuncy in #6793
perf: pin Tantivy main tip and share vector bounds by @walter-woodall in #7077
test: reduce Stressgres bootstrap data for Antithesis by @philippemnoel in #7086
perf: avoid repeated full benchmark publication checkouts by @rebasedming in #7087
fix: Fix filter pushdown in RankJoinExec by @stuhood in #7085
docs: explain Top K pitfalls with subqueries, collations, and computed sorts by @barbarj in #7082
fix: upgrade zip to resolve archive parsing advisory by @philippemnoel in #7094
perf: update Tantivy for vector query preparation by @rebasedming in #7096
perf: late materialization for array fast fields by @devdattatalele in #6869
chore: Consolidate property tests for the aggregate scan by @stuhood in #7099
fix: detach worker queues before destroying parallel context by @rebasedming in #7106
chore: Generate Dockerfiles for v0.26.4 by @paradedb-github-bot[bot] in #7111
fix: authenticate benchmark metric staging by @philippemnoel in #7113
fix: enforce permissions on standalone index reads by @philippemnoel in #7023
fix: match every row for an empty <> ALL array in pushdown by @MannXo in #7100
fix: leave NULL comparison values to PostgreSQL in pushdown by @MannXo in #7102
perf: skip NULL members in hash-join InList pushdown by @ygxio in #7092
ci: baseline Cohere shared buffer hits per query by @rebasedming in #7121
fix: reclaim merge entry segment ID pages by @rebasedming in #7122
perf: pin Tantivy tip for five-list router patience by @walter-woodall in #7116
fix: omit graph dependencies from Stressgres runtime image by @philippemnoel in #7129
ci: Use an unlinked author for benchmark commits by @philippemnoel in #7131
fix: raise on integer overflow in Join Scan arithmetic by @MannXo in #7103
fix: raise on float division by zero in Join Scan by @MannXo in #7104
fix: preserve array values in DISTINCT and GROUP BY by @kumarUjjawal in #7051
fix: give heap filters the fetched tuple's ctid and tableoid by @BilalAtique in #7067
fix: pin workflow actions to commit SHAs by @philippemnoel in #7125
chore: Fix clash between #6869 and #7051 by @stuhood in #7140
fix: isolate release preparation from signing and publishing by @philippemnoel in #7126
fix: sum int8 exactly on the DataFusion aggregate path by @MannXo in #7118
chore: move in-tree benchmarks to the consolidated S3 bucket by @jamessewell in #6980
ci: replace benchmark AWS access keys with scoped OIDC roles by @philippemnoel in #7151
ci: Prune benchmark reports during monthly Pages compaction by @philippemnoel in #7157
ci: Run all benchmark suites nightly by @philippemnoel in #7156
ci: pin Docker smoke tests to working runner AMIs by @philippemnoel in #7176
fix: lock partition heaps before indexes during diagnostics by @mithuncy in #7069
ci: add MS MARCO heap snapshot generation by @rebasedming in #7180
fix: handle NUL bytes in MS MARCO heap imports by @rebasedming in #7182
fix: parallelize and instrument MS MARCO heap loading by @rebasedming in #7185
fix: route sum(int8) to the DataFusion aggregate backend by @MannXo in #7143
fix: preserve exact numeric literals in aggregate filters by @philippemnoel in #7188
ci: fix Cohere benchmark S3 dataset path by @walter-woodall in #7191
ci: Pin extension image smoke tests to working runner AMIs by @philippemnoel in #7192
chore(deps): bump actions/create-github-app-token from 3.2.0 to 3.3.0 in /.github/workflows in the github-actions-dependencies group across 1 directory by @dependabot[bot] in #7155
ci: enable monthly dependency updates by @philippemnoel in #7193
chore(deps): bump the cargo-dependencies group with 17 updates by @dependabot[bot] in #7194
ci: omit unused ID from Cohere search index by @rebasedming in #7190
fix: count array rows in aggregates and windows by @kumarUjjawal in #7053
fix: keep NULL rows out of negated heap filters in Base Scan by @niladrix719 in #7173
fix: apply the trim filter before the token length filter by @hsusul in #7139
fix: fall back for string_agg with a non-constant separator by @niladrix719 in #7161
perf: flatten grouped aggregate results in one pass by @Sque3ze in #7098
feat: periodically sweep underfilled mutable segments by @rebasedming in #6797
docs: Remove unused PostHog integration by @philippemnoel in #7204
ci: Reserve worker slots for benchmark parallelism by @rebasedming in #7208
perf: carry PK-dependent text columns through the aggregate deferred by @ygxio in #7165
fix: keep integer bounds of JSON numeric ranges exact by @MannXo in #7199
fix: align Antithesis SDK with proptest fork by @philippemnoel in #7215
chore: Refactor fast-field and heap column resolution. by @barbarj in #7130
perf: remove vector probe floor and radius overhead by @rebasedming in #7211
feat: replace vector_fields with index-wide quantization by @rebasedming in #6854
ci: run a subset of benchmark queries from a PR comment by @mdashti in #7217
fix: use system Git for macOS release checkouts by @philippemnoel in #7249
chore: Generate Dockerfiles for v0.26.5 by @paradedb-github-bot[bot] in #7253
perf: globally schedule unfiltered vector search by @rebasedming in #7041
perf: defer vector scoring setup for unprobed segments by @rebasedming in #7255
test: pin the segment layout under the order_by_range parallel check by @mdashti in #7246
fix: stop pushing JSONB ? down to an exists query by @MannXo in #7142
test: drop test databases when the test thread exits by @BilalAtique in #7229
fix: return oid 0 as a value, not NULL, from index reads by @niladrix719 in #7231
fix: enforce planner warning modes consistently by @jamessewell in #7050
ci: Pin Docker benchmark runner AMIs by @philippemnoel in #7257
ci: build the benchmark indexes once by @mdashti in #7226
feat: Add support for joins ordered by vector distance by @stuhood in #7137
feat: integrate merged Tantivy vector routing and storage by @walter-woodall in #7128
ci: Restore rolling RunsOn image by @allyn-bottorff in #7269
ci: Enable RunsOn git and apt caches by @philippemnoel in #7274
chore: Remove the v1 API documentation to avoid agentic confusion by @jamessewell in #7043
fix: decode NUMERIC columns in join filters by @niladrix719 in #7162
feat(vector): report router kind and per-level centroid counts in vector_info by @walter-woodall in #6883
perf(vector): quantize the IVF router by default by @RuchirRaj in #7287
chore: Run datafusion-distributed over gRPC by @stuhood in #7108
perf: keep the rank join when a leaf carries a pushed filter such as a row-level security policy by @mithuncy in #7298
feat: add adaptive vector stopping within max_probe budgets by @rebasedming in #7279
fix: Apply merge backpressure to the index-wide segment count by @rebasedming in #7273
chore: Fix conflict between #7108 and #7298. by @stuhood in #7309
chore: Preserve benchmark cluster GUCs across heap snapshot restores. by @stuhood in #7310
fix: polish RankJoin eligibility, planner warnings, and parameterized score projections by @stuhood in #7305
chore: Generate Dockerfiles for v0.26.6 by @paradedb-github-bot[bot] in #7326
ci: Update Postgres.app extension paths in workflow by @tbussmann in #7327
ci: Authenticate benchmark publication commits by @philippemnoel in #7335
ci: Ignore SQLite advisory for unused SQLx driver by @philippemnoel in #7336
ci: add MS MARCO vector benchmark baseline by @rebasedming in #7320
fix: Prevent partitioned segment skew by using ctid as a fallback by @barbarj in #7271
fix: bypass planner hooks during mutable sweeps by @rebasedming in #7301
perf: cache vector rotation plans across queries by @rebasedming in #7339
feat: add synonym indexes on pdb.synonym_rule columns by @mdashti in #7290
chore: Split pg_search into 9 crates. by @stuhood in #7318
feat(vector): share global routing across parallel scans and joins by @rebasedming in #7295
docs: redirect retired third-party extension guide by @philippemnoel in #7357
docs: Put breaking changes first in release notes by @philippemnoel in #7359
ci: add Hacker News vector benchmarks by @rebasedming in #7345
chore: update clustering deps for geometric target_leaf_size by @rebasedming in #7346
ci: accept setting-driven filters in the Cohere plan check by @RuchirRaj in #7363
fix: correct Tantivy revision after clustering update by @rebasedming in #7365
chore: Upgrade to pgrx 0.20.0-beta.0 by @eeeebbbbrrrr in #7328
chore: Fix docs after vector updates. by @stuhood in #7371
perf(vector): doc-ordered exact rows for quantized IVF (.vec V6) by @RuchirRaj in #7275
perf(vector): replicate each vector into a second IVF cluster by @RuchirRaj in #7277
fix: restore IVF split logic while keeping target_leaf_size by @rebasedming in #7377
perf: avoid unnecessary vector read cost lookups by @rebasedming in #7383
ci: run vector benchmarks with shared Docker setup by @rebasedming in #7391
ci: stagger daily benchmark workflows by @philippemnoel in #7392
perf: keep the rank join for row-level security policies of any shape and score them 0.0 in every plan by @mithuncy in #7366
ci: report router quality in the Cohere benchmark by @RuchirRaj in #7356

New Contributors

@sainad2222 made their first contribution in #6461
@wisper-dev made their first contribution in #6727
@DeviousCardi made their first contribution in #6352
@abelmarnk made their first contribution in #6373
@devcool20 made their first contribution in #6789
@kumarUjjawal made their first contribution in #6978
@Yanghanrui666 made their first contribution in #6895
@ygxio made their first contribution in #6707
@Sunny-200 made their first contribution in #6877
@pavanmanishd made their first contribution in #6998
@puppylpg made their first contribution in #7066
@niladrix719 made their first contribution in #7173
@hsusul made their first contribution in #7139
@Sque3ze made their first contribution in #7098
@allyn-bottorff made their first contribution in #7269
@tbussmann made their first contribution in #7327

Full Changelog: v0.26.0...v0.27.0-rc.0