Core Library
- feat: add incremental lag (attribution window) for datetime, int, and float cursors by @donotpush in #1957
- LanceDB - (1) support merge key to merge chunked documents correctly - removes orphaned chunks (2) huge performance upgrade by loading data via arrow by @Pipboyguy in #1620
- Move
exclude_keys()todlt.common.utilsby @burnash in #1966 - Fix BigQueryLoadJob hiding root cause exception by @xneg in #1992
- loads secrets from colab userdata and steamlit + bugfixes by @rudolfix in #1994
- Fix pagination issue in
JSONResponseCursorPaginatorwith empty string cursor value by @kang8 in #2016 - fix: if name of distribution is None by @senickel in #2024
- allows to pass default values when writing specs by @rudolfix in #2018
- enable
deltapartitioning on arrow normalizer load id by @jorritsandbrink in #2022 - add session token to duckdb s3 secret by @jorritsandbrink in #2007
- Add user agent for Databricks by @VioletM in #1987
- Fix an incorrect missing dependency error by @burnash in #2001
- fix resource level max_table_nesting and normalizer performance tuning by @sh-rp in #2026
- move default pipelines of cores sources into source folders by @sh-rp in #1888
- duckdb filesystem custom secrets by @sh-rp in #2017
- allows for empty dataset clickhouse by @rudolfix in #2045
- add GCP default credential handling for
deltatable format by @jorritsandbrink in #2048 - enables merges for bigquery autodetect schema by @sh-rp in #2035
- logs warning if deduplication state is large by @willi-mueller in #1877
- Add core sources extras to requirements in
dlt initby @burnash in #2028 - Fix merge write disposition for pyarrow and ClickHouse by @burnash in #2042
Experimental interfaces
dlt dataset public interface and docs coming next week.
Docs
- Updated databricks destination documentation by @dat-a-man in #1984
- Docs: fix capitalization of some terms, fix typos by @burnash in #1988
- fix typo by @mariarice15 in #1995
- Fix Zendesk example: make test resilient to data changes by @burnash in #1999
- fix s3 credentials environment variable names by @seunggs in #2010
- remove ga add tm by @alexanderfifefd in #2008
- Super fast snippet linting & type checking by @sh-rp in #2019
- Fix the deprecation warning in
.common.configuration.containerby @burnash in #2025 - Added deploy with modal. by @dat-a-man in #1805
- Updated google cloud function documentation by @dat-a-man in #2034
- add warning for large delta memory footprint on filesystem docs page by @sh-rp in #2036
- simplify advanced section by @kning in #2037
- Added docs on how to deploy a pipeline using Google Cloud run by @dat-a-man in #2038
- Format Delta table section in the filesystem destination by @burnash in #2057
- Docs: add table formats to the sidebar by @burnash in #2060
New Contributors
- @xneg made their first contribution in #1992
- @seunggs made their first contribution in #2010
- @alexanderfifefd made their first contribution in #2008
- @kang8 made their first contribution in #2016
- @senickel made their first contribution in #2024
- @kning made their first contribution in #2037
Full Changelog: 1.3.0...1.4.0