github databricks/dbt-databricks v1.13.0

5 hours ago

Features

  • Merge tblproperties additively across config hierarchy levels instead of replacing parent values (thanks @canbekley!) (#1353)
  • Add opt-in invocation telemetry for eligible commands, enabled with connection_parameters.enable_dbt_telemetry, that reports aggregate POST_PARSE model configuration statistics and POST_RUN outcomes when the workspace server flag allows it or connection_parameters.force_enable_dbt_telemetry is set by @saishreeeee in #1647, #1682, #1648, #1665
  • BREAKING for some view models: Views use one materialization regardless of use_materialization_v2, so with the flag off view_update_via_alter now alters Unity Catalog views in place and fails on Hive metastore views, grants apply after tags, and view replacement calls get_create_view_as_sql overrides by @sd-db in #1737

Fixes

  • Stop view_update_via_alter failing with UNRESOLVED_COLUMN when a query change drops a column whose description also changed, and keep column comments after query changes on views inheriting a non-Delta file_format by @sd-db in #1736
  • Enforce model contracts and reject column masks and row filters when view_update_via_alter alters a view in place or finds no changes, matching CREATE by @sd-db in #1736
  • Replace views with CREATE OR REPLACE VIEW when use_safer_relation_operations is on, including views inheriting file_format: parquet, so upstream column drops or renames no longer fail with UNRESOLVED_COLUMN (thanks @shantanutheone!) (#1733 resolves #1714)
  • Skip transaction: false hooks with a warning by default, or run them without COMMIT when use_non_transactional_hooks is enabled; the inherited function/UDF materialization remains unchanged by @sd-db in #1693 (resolves #1549)
  • Reapply view queries containing * when view_update_via_alter is enabled so upstream column additions appear, while unchanged explicit projections stay no-ops by @sd-db in #1720 (resolves #1356)
  • Let Predictive Optimization manage models with auto_liquid_cluster instead of running synchronous OPTIMIZE after materialization by @sd-db in #1658 (resolves #1655)
  • Skip unnecessary Unity Catalog constraint metadata queries for incremental models without enforced contracts by @sd-db in #1643 (resolves #1641)
  • Recreate materialized views when query schema drifts, honoring on_configuration_change by @sd-db in #1621 (resolves #1359)
  • Warn when documented columns are missing from table and incremental models, and honor persist_docs.columns for V2 column comments (thanks @r-jais!) (#1563 resolves dbt-labs/dbt-adapters#1690)
  • Warn when documented columns are missing after creating views, materialized views, or streaming tables with persist_docs.columns enabled (thanks @r-jais!) (#1615 resolves #1399)
  • Log that dbt clone skips ephemeral models instead of reporting them missing from the state manifest (thanks @ajhlee-dbt!) (#1663 resolves dbt-labs/dbt#14474)

Under the Hood

  • Raise the dbt-core upper bound to <1.12.7 to include dbt-core 1.12.6 by @sd-db in #1739
  • Raise the databricks-sql-connector floor to >=4.2.2 so the telemetry server check runs once per host instead of once per connection by @saishreeeee in #1682
  • Remove unused internal credential, pipeline, and connection logging-event classes by @sd-db in #1547
  • Fetch table tags through the Unity Catalog Entity Tag Assignments API to reduce metadata-query rate limiting, falling back to information_schema when the token cannot call the API by @sd-db in #1603

Full Changelog: v1.12.8...v1.13.0

Don't miss a new dbt-databricks release

NewReleases is sending notifications on new releases.