dlt 1.26.0 Release Notes
Breaking Changes
- Incremental external scheduler now raises instead of silently warning (#3877 @rudolfix) — Untyped/non-coercible cursor values now raise
JoinSchedulerError; missing intervals raiseExternalSchedulerNotAvailable. Resources withallow_external_schedulers=Truethat previously fell back to dlt state will now fail. This is a bugfix that corrects previously incorrect behavior.
Highlights
dlt.Relation.join(...)(#3590 @Travior) — Adds ajoin()method ondlt.Relationbased on the normalizer and table references, enabling fluent relational composition over datasets.- Extended Snowflake query tags (#3759 @Travior) — Snowflake query tagging is extended beyond load jobs to broader dlt operations: storage setup, schema/state reads, schema updates, load execution, load completion, and table drops.
TJobQueryTagsis generalized toTQueryTagswith a newoperationfield (with a compatibility export). - Time interval context for incrementals (#3877 @rudolfix) — New
dlt.current.interval()returns the active(start, end)interval orNone, backed by an injectableTimeIntervalContextwith optionalallow_external_schedulersoverride and auto-detection from env vars / Airflow.
Core Library
dlt.Relation.join(...)(#3590 @Travior) — see Highlights.- Extended Snowflake query tags beyond load jobs (#3759 @Travior) — see Highlights.
- Time interval context for incrementals (#3877 @rudolfix) — see Highlights.
- Destination-aware incremental SQL filter (#3877 @rudolfix) — New
dlt/extract/incremental/sql.py(to_sqlglot_filter) honorstimestamp_precision,supports_tz_aware_datetime_in_cast, and sqlite quirks; works on bound and unbound incrementals. start_valuepersisted in incremental state (#3877 @rudolfix) — Only written when rows actually arrive, so it is no longer advanced silently on empty runs.uuid_to_stringPyArrow fast path (#3877 @rudolfix) — Numpy-vectorized with a pure-Python fallback; pyarrow ≥ 24arrow.uuidextension arrays are coerced to canonical strings, and UUID columns under pyarrow < 24 also take the fast path.- Custom metrics emitted even when a resource produces no data (#3877 @rudolfix) —
dlt.current.resource_metrics()counters are no longer dropped when every item is filtered out. - TypedDict validator honors
NotRequired[T](#3877 @rudolfix) — Via__required_keys__. - sqlglot
"dremio"dialect literal (#3877 @rudolfix) — Added toTSqlGlotDialect. - Opt-in naming-convention check in
Schema.unify_schemas()(#3898 @burnash) — The naming-convention check inSchema.unify_schemas()is now opt-in; also drops the max_length tests workaround. - Fix(clickhouse): correct metadata sort keys to avoid full scans (#3851 @anuunchin) — Resolves #3806 by correcting metadata sort keys so ClickHouse no longer performs full scans.
- Fix(pyarrow): preserve string encoding for UUID columns under pyarrow 24+ (#3894 @burnash) — Fixes #3893 so UUID columns keep their string encoding under pyarrow 24+.
- Fix: clearer error from
dlt.attach()when pipeline cannot be restored (#3890 @bjoaquinc) — RewritesCannotRestorePipelineExceptionmessages to name required inputs, show a concretedlt.attach(...)example, and offerdlt.dataset()as a lighter alternative; suppresses a redundant inner exception in tracebacks. - Fix: closed-pipe race in
iter_std(#3877 @rudolfix) — Reader threads swallowValueError/OSErrorand always close the queue. - Fix: active profile shown in
ConfigFieldMissingException(#3877 @rudolfix)
Docs
- Improved dashboard docs (#3484 @VioletM)
- Updated documentation for pg_replication (#3900 @dat-a-man) — Adds an "Alternative: CDC with Debezium" section linking to the Debezium + dlt demo.
- Snowflake marketplace listing link (#3888 @kaliole)
- Quality-checker fixes to
pipeline.md(#3885 @ShreyasGS) — Tense, contractions, and grammar cleanups via Harper + Vale Google Developer Docs style. - Remove playground page and associated CI (#3883 @zilto) — Streamlines the "Getting Started" section; interactive material remains on Google Colab and marimo molab.
- Pydantic Logfire export example (#3882 @zilto)
New Contributors
- @bjoaquinc made their first contribution in #3890