github dlt-hub/dlt 1.26.0

7 hours ago

dlt 1.26.0 Release Notes

Breaking Changes

  1. Incremental external scheduler now raises instead of silently warning (#3877 @rudolfix) — Untyped/non-coercible cursor values now raise JoinSchedulerError; missing intervals raise ExternalSchedulerNotAvailable. Resources with allow_external_schedulers=True that previously fell back to dlt state will now fail. This is a bugfix that corrects previously incorrect behavior.

Highlights

  • dlt.Relation.join(...) (#3590 @Travior) — Adds a join() method on dlt.Relation based on the normalizer and table references, enabling fluent relational composition over datasets.
  • Extended Snowflake query tags (#3759 @Travior) — Snowflake query tagging is extended beyond load jobs to broader dlt operations: storage setup, schema/state reads, schema updates, load execution, load completion, and table drops. TJobQueryTags is generalized to TQueryTags with a new operation field (with a compatibility export).
  • Time interval context for incrementals (#3877 @rudolfix) — New dlt.current.interval() returns the active (start, end) interval or None, backed by an injectable TimeIntervalContext with optional allow_external_schedulers override and auto-detection from env vars / Airflow.

Core Library

  • dlt.Relation.join(...) (#3590 @Travior) — see Highlights.
  • Extended Snowflake query tags beyond load jobs (#3759 @Travior) — see Highlights.
  • Time interval context for incrementals (#3877 @rudolfix) — see Highlights.
  • Destination-aware incremental SQL filter (#3877 @rudolfix) — New dlt/extract/incremental/sql.py (to_sqlglot_filter) honors timestamp_precision, supports_tz_aware_datetime_in_cast, and sqlite quirks; works on bound and unbound incrementals.
  • start_value persisted in incremental state (#3877 @rudolfix) — Only written when rows actually arrive, so it is no longer advanced silently on empty runs.
  • uuid_to_string PyArrow fast path (#3877 @rudolfix) — Numpy-vectorized with a pure-Python fallback; pyarrow ≥ 24 arrow.uuid extension arrays are coerced to canonical strings, and UUID columns under pyarrow < 24 also take the fast path.
  • Custom metrics emitted even when a resource produces no data (#3877 @rudolfix) — dlt.current.resource_metrics() counters are no longer dropped when every item is filtered out.
  • TypedDict validator honors NotRequired[T] (#3877 @rudolfix) — Via __required_keys__.
  • sqlglot "dremio" dialect literal (#3877 @rudolfix) — Added to TSqlGlotDialect.
  • Opt-in naming-convention check in Schema.unify_schemas() (#3898 @burnash) — The naming-convention check in Schema.unify_schemas() is now opt-in; also drops the max_length tests workaround.
  • Fix(clickhouse): correct metadata sort keys to avoid full scans (#3851 @anuunchin) — Resolves #3806 by correcting metadata sort keys so ClickHouse no longer performs full scans.
  • Fix(pyarrow): preserve string encoding for UUID columns under pyarrow 24+ (#3894 @burnash) — Fixes #3893 so UUID columns keep their string encoding under pyarrow 24+.
  • Fix: clearer error from dlt.attach() when pipeline cannot be restored (#3890 @bjoaquinc) — Rewrites CannotRestorePipelineException messages to name required inputs, show a concrete dlt.attach(...) example, and offer dlt.dataset() as a lighter alternative; suppresses a redundant inner exception in tracebacks.
  • Fix: closed-pipe race in iter_std (#3877 @rudolfix) — Reader threads swallow ValueError/OSError and always close the queue.
  • Fix: active profile shown in ConfigFieldMissingException (#3877 @rudolfix)

Docs

  • Improved dashboard docs (#3484 @VioletM)
  • Updated documentation for pg_replication (#3900 @dat-a-man) — Adds an "Alternative: CDC with Debezium" section linking to the Debezium + dlt demo.
  • Snowflake marketplace listing link (#3888 @kaliole)
  • Quality-checker fixes to pipeline.md (#3885 @ShreyasGS) — Tense, contractions, and grammar cleanups via Harper + Vale Google Developer Docs style.
  • Remove playground page and associated CI (#3883 @zilto) — Streamlines the "Getting Started" section; interactive material remains on Google Colab and marimo molab.
  • Pydantic Logfire export example (#3882 @zilto)

New Contributors

Don't miss a new dlt release

NewReleases is sending notifications on new releases.