dbt-labs/dbt-utils 1.0.0-rc1

on GitHub

This is the first release candidate for dbt utils 1.0. A full migration guide will accompany the final release, but here is the changelog:

New

• New macro: get_single_value()
• New macro: safe_divide()
• New test: not_empty_string

Enhancements

• group_by_columns in some tests
• Able to prevent quote_identifiers in star()

Fixes

• union now includes/excludes columns case-insensitively
• slugify prefixes an underscore when the first char is a digit
• expression_is_true doesn’t output * unless storing failures
• star() only returns a * when no columns are found if running under the dbt compile command or execute=false mode. Returning * in these contexts allows SQLFluff's linting to work, but returning nothing under standard conditions ensure that you don't accidentally pull all your columns if you meant to exclude them all.
• recency test can now be configured to ignore the time component, useful when your tested column is a date instead of a datetime.

Deprecated

• The expression_is_true test no longer has a dedicated condition argument. Instead, use where which is now available natively to all tests
• The unique_where and not_null_where tests have been removed, because where is now available natively to all tests.
• 🚨 surrogate_key() has been replaced by generate_surrogate_key(). The original treated null values and blank strings the same, which could lead to duplicate keys being created. generate_surrogate_key() does not have this flaw. Compare the surrogate keys calculated for these columns

Changing the calculation method for surrogate keys, even for the better, could have significant consequences in downstream uses (such as snapshots and incremental models which use this column as their unique_key). Because of this, it is possible to opt in to the legacy behaviour by setting the following variable in your dbt project:

#dbt_project.yml
vars:
  surrogate_key_treat_nulls_as_empty_strings: true #turn on legacy behaviour

By creating a new macro instead of updating the behaviour of the old one, we are requiring all projects who use this macro to make an explicit decision about which approach is better for their context. Our recommendation is that existing users should opt into the legacy behaviour unless you are confident that either a) your surrogate keys never contained nulls, or b) your surrogate keys are not used for incremental models, snapshots or other stateful artifacts and so can be regenerated with new values without issue.
Warning to package maintainers: you can not assume one behaviour or the other, as it can be enabled/disabled by the end user.

• Deprecation: current_timestamp() replaced by dbt.current_timestamp(). Note that Postgres and Snowflake’s implementation of current_timestamp() differs between the old dbt_utils one and the new dbt one (full details here). For those adapters, use dbt.current_timestamp_backcompat() if you need identical, backwards-compatible behaviour. This discrepancy will hopefully be reconciled in some future version of dbt Core (maybe 2.0?)
• All other cross-db macros have moved to dbt namespace with no changes necessary other than swapping out dbt_utils. for dbt.; see https://docs.getdbt.com/reference/dbt-jinja-functions/cross-database-macros
• insert_by_period materialization moved to experimental repo
• safe_add() only works with a list of arguments
• deduplicate() changes:
  • group_by argument is removed, replaced by partition_by
  • relation_alias is removed, replaced by passing the alias directly to the relation argument
  • order_by is now mandatory. Pass a static value like 1 if you don’t care how they are deduplicated
• table argument has been removed from unpivot. Use relation instead

Full Changelog: 0.9.6...1.0.0-rc1