dbt-spark 1.5.0 - April 27, 2023
Features
- Support for data types constraints in Spark following the dbt Core feature #6271 (#558)
- Enforce contracts on models materialized as tables and views (#639, #654)
- Modify adapter to support unified constraint fields (#655)
- Modify order of columns in create_table_as to match contract (#671)
Fixes
- Support for iceberg v2 tables. Added ability to use multiple join conditions to allow for multiple columns to make a row distinct. (#294)
- Use take() instead of collect on dataframe to improve the performance (#526)
- add merge_exclude_columns tests (#00)
- Fix pyodbc type_code -> data_type conversion (#665)
- Fixed issue where table materialization was not always properly refreshing for non-admin users on Databricks (#725)
Under the Hood
Dependencies
- Allow thrift 0.16.0 (#605)
- Update ipdb requirement from ~=0.13.11 to ~=0.13.13 (#677)
- Update wheel requirement from ~=0.38 to ~=0.40 (#680)
- Update pre-commit requirement from ~=2.21 to ~=3.2 (#687)
- Update wheel requirement from ~=0.38 to ~=0.40 (#680)
- Bump mypy from 1.0.1 to 1.1.1 (#675)
- Update types-pytz requirement from ~=2022.7 to ~=2023.2 (#697)
- Update pytz requirement from ~=2022.7 to ~=2023.2 (#696)