PR #164 includes the following updates:
Schema Data Changes
3 total changes • 3 possible breaking changes
To prevent potential errors from naming and materialization updates, a
--full-refreshis required after upgrading.
| Data Models | Change Type | Old | New | Notes |
|---|---|---|---|---|
fivetran_platform__audit_table
| Model materialization (only for Databricks SQL Warehouse runtimes) | table | incremental | Added incremental model support for Databricks SQL Warehouses using the merge strategy. Previously, incremental support was limited to Databricks All Purpose Clusters via the insert_overwrite strategy. This update extends incremental functionality to SQL Warehouses, enabling more efficient model builds.
|
fivetran_platform__audit_table
| New column | write_to_table_start_day
| Changed the column partitioned on from sync_start_day to the new write_to_table_start_day. The previous column could contain null values, which are not ideal for partitioning and may lead to unexpected behavior in incremental models.
| |
fivetran_platform__audit_table
| Deleted column | sync_start_day
| No longer in use given the above. |
Bug Fixes
- Updated
fivetran_platform__mar_table_historyto include consumption records do not have an associated active connection and/or destination.- As a result this table may now contain additional records that were previously excluded.
- For more details, see the corresponding DECISIONLOG entry.
dbt Fusion Compatibility Updates
- Updated package to maintain compatibility with dbt-core versions both before and after v1.10.6, which introduced a breaking change to multi-argument test syntax (e.g.,
unique_combination_of_columns). - Temporarily removed unsupported tests to avoid errors and ensure smoother upgrades across different dbt-core versions. These tests will be reintroduced once a safe migration path is available.
- Removed all
dbt_utils.unique_combination_of_columnstests. - Moved
loaded_at_field: _fivetran_syncedunder theconfig:block insrc_fivetran_log.yml.
- Removed all
Under the Hood
- Updated the
is_incremental_compatible()macro to include Databricks SQL Warehouses. - Introduced a new macro,
is_databricks_all_purpose_cluster(), to distinguish between Databricks All Purpose Clusters and SQL Warehouses. - Updated conditions in
.github/workflows/auto-release.yml. - Added
.github/workflows/generate-docs.yml.
Full Changelog: v2.2.2...v2.3.0