Patch Changes
-
Enhanced LoadSubsetOptions with separate cursor expressions and offset for flexible pagination. (#960)
⚠️ Breaking Change for Custom Sync Layers / Query Collections:
LoadSubsetOptions.whereno longer includes cursor expressions for pagination. If you have a custom sync layer or query collection that implementsloadSubset, you must now handle pagination separately:- Cursor-based pagination: Use the new
cursorproperty (cursor.whereFromandcursor.whereCurrent) and combine them withwhereyourself - Offset-based pagination: Use the new
offsetproperty
Previously, cursor expressions were baked into the
whereclause. Now they are passed separately so sync layers can choose their preferred pagination strategy.Changes:
- Added
CursorExpressionstype withwhereFrom,whereCurrent, and optionallastKeyproperties - Added
cursortoLoadSubsetOptionsfor cursor-based pagination (separate fromwhere) - Added
offsettoLoadSubsetOptionsfor offset-based pagination support - Electric sync layer now makes two parallel
requestSnapshotcalls when cursor is present:- One for
whereCurrent(all ties at boundary, no limit) - One for
whereFrom(rows after cursor, with limit)
- One for
- Query collection serialization now includes
offsetfor query key generation - Added
truncateevent to collections, emitted when synced data is truncated (e.g., aftermust-refetch) - Fixed
setWindowpagination: cursor expressions are now correctly built when paging through results - Fixed offset tracking:
loadNextItemsnow passes the correct window offset to prevent incorrect deduplication CollectionSubscribernow listens fortruncateevents to reset cursor tracking state
Benefits:
- Sync layers can choose between cursor-based or offset-based pagination strategies
- Electric can efficiently handle tie-breaking with two targeted requests
- Better separation of concerns between filtering (
where) and pagination (cursor/offset) setWindowcorrectly triggers backend loading for subsequent pages in multi-column orderBy queries- Cursor state is properly reset after truncation, preventing stale cursor data from being used
- Cursor-based pagination: Use the new
-
Ensure deterministic iteration order for collections and indexes. (#958)
SortedMap improvements:
- Added key-based tie-breaking when values compare as equal, ensuring deterministic ordering
- Optimized to skip value comparison entirely when no comparator is provided (key-only sorting)
- Extracted
compareKeysutility toutils/comparison.tsfor reuse
BTreeIndex improvements:
- Keys within the same indexed value are now returned in deterministic sorted order
- Optimized with fast paths for empty sets and single-key sets to avoid unnecessary allocations
CollectionStateManager changes:
- Collections now always use
SortedMapforsyncedData, ensuring deterministic iteration order - When no
comparefunction is provided, entries are sorted by key only
This ensures that live queries with
orderByandlimitproduce stable, deterministic results even when multiple rows have equal sort values. -
Enhanced multi-column orderBy support with lazy loading and composite cursor optimization. (#926)
Changes:
- Create index on first orderBy column even for multi-column orderBy queries, enabling lazy loading with first-column ordering
- Pass multi-column orderBy to loadSubset with precise composite cursors (e.g.,
or(gt(col1, v1), and(eq(col1, v1), gt(col2, v2)))) for backend optimization - Use wide bounds (first column only) for local index operations to ensure no rows are missed
- Use precise composite cursor for sync layer loadSubset to minimize data transfer
Benefits:
- Multi-column orderBy queries with limit now support lazy loading (previously disabled)
- Sync implementations (like Electric) can optimize queries using composite indexes on the backend
- Local collection uses first-column index efficiently while backend gets precise cursor
-
Updated dependencies [
52c29fa]:- @tanstack/db-ivm@0.1.14