PR #114 includes the following updates:
Breaking Changes
⚠️ Since the following changes are breaking, we recommend running a
--full-refreshafter upgrading to this version.
- For Bigquery and Databricks destinations, updated the
partition_byconfig to coordinate with the filter used in the incremental logic. - For Snowflake destinations, added a
cluster_byconfig for performance.
Feature Updates
- Updated incremental logic for
fivetran_platform__audit_tableso that it looks back 7 days to catch any late arriving records. - Updated JSON parsing logic in the following models to prevent run failures when incoming JSON-like strings are invalid.
fivetran_platform__audit_tablefivetran_platform__audit_user_activityfivetran_platform__connector_daily_eventsfivetran_platform__connector_statusfivetran_platform__schema_changelog
- Updated
fivetran_platform__connector_statusto parse only a subset of themessage_datafield to improve compute.
Under The Hood
- Added macros:
fivetran_log_json_parseto handle the updated JSON parsing.fivetran_log_lookbackfor use infivetran_platform__audit_table.
- Updated seeds to test handling of invalid JSON strings.
Full Changelog: v1.4.3...v1.5.0