Core Library
-
use Airflow (and possibly other) schedulers with dlt resources by @rudolfix in #534
A really cool feature that allows your incremental loading to take date ranges from Airflow schedulers. Do backfilling, incremental loading and relay on Airflow to keep the pipeline state. -
Ignore hints prefixed with 'x-' in table_schema() by @burnash in #525
-
Now our CI works correctly from forks! by @steinitzu in #530
Support for unstructured data!
A really cool data source that let's you ask questions about your PDF documents and stores the answers in any of our destinations. Going from binary blobs through unstrucutred.io, vector databases and LLM queries to ie. duckdb and bigquery. Blobs coming from filesystem, google drive or your inbox (also incrementally) by @AstrakhantsevaAA