Major release
This is a major dlt release (as per our semantic versioninghttps://github.com/dlt-hub/dlt?tab=readme-ov-file#adding-as-dependency). It brings several interesting new features like: schema evolution control, data contracts, deeper Pydantic integration, parametrized destinations, improvements to parallelism and data lineage + many more
There are no significant breaking changes, but minor ones exist, please refer to #763 for details
Core Library
- Parametrized destinations - import destinations from
dlt.destinationsmodule and instantiate them: by @steinitzu in #746 - schema and data contracts by @sh-rp in #594
- load package id in extract step by @rudolfix in #790
- named destinations: configure many destinations with different names by @sh-rp in #783
- rich tracing information from pipeline steps (extract, normalize, load) by @rudolfix in #801
- adds exception stack to pipeline trace by @rudolfix in #806
- fixed attribute check: getuid -> geteuid by @jorritsandbrink in #823
- allows to run parallel pipelines in separate threads by @rudolfix in #813
- 791 test mssql credentialspy is odbc driver 18 dependent by @jorritsandbrink in #834
- adds extract and normalize traces by @rudolfix in #839
Plus some tooling changes
- introduce black formatting by @sh-rp in #583
- Fix: ensure accessor typing does not make static type checker error by @z3z1ma in #785
- Hot fix: add skipifgithubfork to nested_data example by @AstrakhantsevaAA in #802
- Fix Windows lint issue and implement CI lint matrix strategy by @jorritsandbrink in #827
Docs
- documents schema and data contract by @rudolfix in #782
- Added Kinesis documentation. by @dat-a-man in #804
- 788 clarify docs intro by @deanja in #797
- Fix links to source code by @AstrakhantsevaAA in #805
- Clarify docs dev process by @deanja in #809
- Qdrant ingestion pipeline example eg by @hibajamal in #775
- Personio doc: added more endpoints by @AstrakhantsevaAA in #829
New Contributors
- @deanja made their first contribution in #797
- @IlyaFaer made their first contribution in #820
- @jorritsandbrink made their first contribution in #823
Full Changelog: 0.3.25...0.4.1