Migration guide: https://docs.getdbt.com/guides/migration/versions/upgrading-to-dbt-utils-v1.0
Breaking changes:
🚨 surrogate_key() has been replaced by generate_surrogate_key(). You have to make a decision🚨
The original treated null values and blank strings the same, which could lead to duplicate keys being created. generate_surrogate_key() does not have this flaw. If needed, it's possible to opt into the legacy behavior by setting the following variable in your dbt project:
#dbt_project.yml
vars:
surrogate_key_treat_nulls_as_empty_strings: true #turn on legacy behavior
Our recommendation is that existing users should opt into the legacy behaviour unless you are confident that either:
- your surrogate keys never contained nulls, or
- your surrogate keys are not used for incremental models, snapshots or other stateful artifacts and so can be regenerated with new values without issue.
dbt_utils.current_timestamp() is replaced by dbt.current_timestamp()
Postgres and Snowflake’s implementation of dbt.current_timestamp() differs from the old dbt_utils one (full details here). If you use Postgres or Snowflake and need identical backwards-compatible behaviour, use dbt.current_timestamp_backcompat()
.
Cross-db macros
All other cross-db macros have moved to the dbt namespace, with no changes necessary other than replacing dbt_utils.
with dbt.
. Review the cross database macros documentation for the full list, or the migration guide for a find-and-replace regex
Other functionality now native to dbt Core:
- The
expression_is_true
test no longer has a dedicated condition argument. Instead, use where which is available natively to all tests - For the same reason, the deprecated
unique_where
andnot_null_where
tests have been removed
insert_by_period removed
The insert_by_period materialization has been moved to the experimental-features repo. To continue to use it, add the below to your packages.yml file:
packages:
- git: https://github.com/dbt-labs/dbt-labs-experimental-features
subdirectory: insert_by_period
revision: XXXX #optional but highly recommended. Provide a full git sha hash, e.g. 1c0bfacc49551b2e67d8579cf8ed459d68546e00. If not provided, uses the current HEAD.
New features
- get_single_value() — An easy way to pull a single value from a SQL query, instead of having to access the [0][0]th element of a run_query result.
- safe_divide() — Returns null when the denominator is 0, instead of throwing a divide-by-zero error.
- New not_empty_string test — An easier wrapper than using expression_is_true to check the length of a column.
Enhancements
- Many tests are more meaningful when you run them against subgroups of a table. For example, you may need to validate that recent data exists for every turnstile instead of a single data source being sufficient. Add the new group_by_columns argument to your tests to do so. Review this article by the test's author for more information.
- With the addition of an on-by-default quote_identifiers flag in the star() macro, you can now disable quoting if necessary.
Fixes
- union() now includes/excludes columns case-insensitively
- slugify() prefixes an underscore when the first char is a digit
- The expression_is_true test doesn’t output * unless storing failures, a cost improvement for BigQuery.
New Contributors
- @jeremyyeo made their first contribution in #587
- @bakerbryce made their first contribution in #676
- @Maayan-s made their first contribution in #647
- @elyobo made their first contribution in #687
- @alexmalins made their first contribution in #688
- @zachoj10 made their first contribution in #692
- @colin-rogers-dbt made their first contribution in #694
- @CR-Lough made their first contribution in #696
- @ian-fahey-dbt made their first contribution in #718
- @WillAyd made their first contribution in #727
Full Changelog: 0.9.6...1.0.0