dlt-hub/dlt 1.27.0 on GitHub

dlt 1.27.0 Release Notes

Breaking Changes

workspace extra removed and dlthub command split out (#3929 @rudolfix) — The workspace extra is gone; users should install marimo, pyarrow, ibis, fastmcp, and other dependencies directly. Part of dev tooling moved to a plugin: dlt dashboard, dlt pipeline ... show, dlt pipeline ... mcp now require pip install dlt[hub]. dlt ai was moved to dlthub ai.

Highlights

Native Polars DataFrame and LazyFrame support in resources (#3837 @AyushPatel101) — Polars DataFrames and LazyFrames can now be yielded directly from @dlt.resource without manual conversion. Auto-detected and routed through the Arrow extraction pipeline; LazyFrames are auto-collected before conversion.
Databricks Zerobus loading (#3904 @jorritsandbrink) — New databricks_adapter(my_resource, insert_api="zerobus") enables loading via the Databricks Zerobus SDK into Delta tables. API mirrors bigquery_adapter; supported for append write disposition.
Incremental filtering for dlt.Relation (#3889 @burnash) — Apply incremental filters directly on dlt.Relation, enabling efficient incremental reads from datasets.
dlthub command split (#3929 @rudolfix) — Reorganizes _workspace modules and splits dev tooling (dashboard, mcp, ai) into a dedicated dlt[hub] plugin. See Breaking Changes above.

Core Library

Native Polars DataFrame / LazyFrame support (#3837 @AyushPatel101) — See Highlights.
Databricks Zerobus loading (#3904 @jorritsandbrink) — See Highlights.
Incremental filtering for dlt.Relation (#3889 @burnash) — See Highlights.
dlthub command split (#3929 @rudolfix) — See Highlights / Breaking Changes.
lance destination REST Namespace support (experimental) (#3908 @jorritsandbrink) — Adds experimental REST Namespace support to the lance destination, currently only validated against an ephemeral in-memory Lance REST Namespace proxy.
-y / --yes flag to bypass non-interactive prompts (#3910 @anuunchin) — New flag auto-accepts all confirm() prompts via a dedicated ALWAYS_CONFIRM flag in echo.py, without leaking into prompt() or text_input(). Resolves #3592.
Control dashboard section opening from URL (#3686 @djudjuu) — Lets external tools / agents pre-open a specific section of the dashboard via URL, useful for guiding users to particular pipeline state.
Fix: Pydantic annotation metadata handling across 2.10–2.13 (#3863 @Travior) — Restores discriminated RootModel handling and preserves root-level Annotated[...] metadata after Pydantic 2.13 moved the discriminator off root_field.annotation.
Fix: correctly materialize empty tables for multi-table resources (#3901 @anuunchin) — Tracks empty tables (not resources) so multi empty-table materialization works correctly. Resolves #3840.
Fix: scd2 unmapped insert for nested tables (#3812 @deschman) — Generates an explicit column list (excluding SCD2 from/to metadata) for the staging-to-destination insert, fixing DATATYPE_MISMATCH errors on Spark/Databricks ANSI mode when schemas drift. Fixes #3811.
Fix(clickhouse): replica-safe merge delete temp tables (#3824 @RedZapdos123) — Detects the destination table engine and uses ReplicatedMergeTree / SharedMergeTree for merge delete temp tables, preventing duplicates on replicated/shared MergeTree deployments. Closes #3797.
Fix(databricks): replace deprecated _user_agent_entry (#3935 @xodn348) — Switches DatabricksCredentials.to_connector_params() to the non-prefixed user_agent_entry, silencing the deprecation warning printed on every connection. Closes #3934.
Fix: with_load_id breaking on non-root sibling branches (#3878 @Travior)
Fix: schema naming convention respects max_length param (#3826 @aditypan) — Removes spurious length appending in schema naming when max_length is provided. Fixes #3816.
Fix: propagate dataset_name to job metrics in load step (#3808 @rudolfix)

Docs

Docs structure overhaul (#3884 @zilto) — Reorganizes the documentation site structure.
OSS / paid split documentation (#3745 @VioletM)
Cookbook: Arize Phoenix example (#3914 @zilto)
Improve docs around join usage (#3902 @Travior)
Misc docs fixes (#3909 @zilto)

Chores

CI from fork (#3897 @anuunchin) — Splits CI so secret-requiring tests run via a new fork_tests_with_secrets.yml workflow gated by the ci from fork label and a maintainer approval in the fork-ci environment. Resolves #3850.
Workspace tests setup fixes + CLI docs refresh (#3948 @rudolfix)
Scope fruit_pipeline fixture to module to avoid duckdb lock race (#3932 @burnash) — Stabilizes the dashboard e2e suite.

New Contributors

@xodn348 made their first contribution in #3935
@RedZapdos123 made their first contribution in #3824
@deschman made their first contribution in #3812