pypi dbt-core 1.8.0
dbt-core v1.8.0
on Python PyPI

11 days ago

dbt-core 1.8.0 - May 09, 2024

Breaking Changes

  • Remove adapter.get_compiler interface (#9148)
  • Move AdapterLogger to adapters folder (#9151)
  • Rm --dry-run flag from 'dbt deps --add-package', in favor of just 'dbt deps --lock' (#9100)
  • move event manager setup back to core, remove ref to global EVENT_MANAGER and clean up event manager functions (#9150)
  • Remove dbt-tests-adapter and dbt-postgres packages from dbt-core (#9455)
  • Update the default behaviour of require_explicit_package_overrides_for_builtin_materializations to True. (#10062)

Features

  • Initial implementation of unit testing (#8287)
  • Unit test manifest artifacts and selection (#8295)
  • Support config with tags & meta for unit tests (#8294)
  • Allow adapters to include package logs in dbt standard logging (#7859)
  • Enable inline csv fixtures in unit tests (#8626)
  • Add drop_schema_named macro (#8025)
  • migrate utils to common and adapters folders (#8924)
  • Move Agate helper client into common (#8926)
  • remove usage of dbt.config.PartialProject from dbt/adapters (#8928)
  • Add exports to SavedQuery spec (#8892)
  • Support unit testing incremental models (#8422)
  • Add support of csv file fixtures to unit testing (#8290)
  • Remove legacy logger (#8027)
  • Unit tests support --defer and state:modified (#8517)
  • Support setting export configs hierarchically via saved query and project configs (#8956)
  • Support source inputs in unit tests (#8507)
  • Use daff to render diff displayed in stdout when unit test fails (#8558)
  • Global config for --target and --profile CLI flags and DBT_TARGET and DBT_PROFILE environment variables. (#7798)
  • Move unit testing to test command (#8979)
  • Support --empty flag for schema-only dry runs (#8971)
  • Support unit tests in non-root packages (#8285)
  • Convert the tests config to data_tests in both dbt_project.yml and schema files. in schema files. (#8699)
  • Make fixture files full-fledged parts of the manifest and enable partial parsing (#9067)
  • Adds support for parsing conversion metric related properties for the semantic layer. (#9203)
  • Package selector syntax for the current package (#6891)
  • In build command run unit tests before models (#9128)
  • Move flags from UserConfig in profiles.yml to flags in dbt_project.yml (#9183)
  • Added hook support for dbt source freshness (#5609)
  • Align with order of unit test output when actual differs from expected (#9370)
  • Added support for external nodes in unit test nodes (#8944)
  • Enable unit testing versioned models (#9344)
  • Enable list command for unit tests (#8508)
  • Integration Test Optimizations (#9498)
  • Accelerate integration tests with caching. (#9498)
  • Cache environment variables (#9489)
  • Support meta at the config level for Metric nodes (#9441)
  • Add cache to SavedQuery config (#9540)
  • Support scrubbing secret vars (#7247)
  • Allow excluding resource types for build, list, and clone commands, and provide env vars (#9237)
  • SourceDefinition.meta represents source-level and table-level meta properties, instead of only table-level (#9766)
  • Allow metrics in semantic layer filters. (#9804)
  • Add wildcard support to the group selector method (#9811)
  • source freshness precomputes metadata-based freshness in batch, if possible (#8705)
  • Better error message when trying to select a disabled model (#9747)
  • Support SQL in unit testing fixtures (#9405)
  • Add require_explicit_package_overrides_for_builtin_materializations to dbt_project.yml flags, which can be used to opt-out of overriding built-in materializations from packages (#10007)
  • add --empty flag to dbt build command (#10026)
  • Ability to silence warnings via warn_error_options (#9644)
  • Allow aliases error for include and warn for exclude in warn_error_options (#9644)
  • Add unit_test: selection method (#10053)

Fixes

  • For packages installed with tarball method, fetch metadata to resolve nested dependencies (#8621)
  • Fix partial parsing not working for semantic model change (#8859)
  • Handle unknown type_code for model contracts (#8877, #8353)
  • Rework get_catalog implementation to retain previous adapter interface semantics (#8846)
  • Add back contract enforcement for temporary tables on postgres (#8857)
  • Add version to fqn when version==0 (#8836)
  • Fix cased comparison in catalog-retrieval function. (#8939)
  • Catalog queries now assign the correct type to materialized views (#8864)
  • Fix compilation exception running empty seed file and support new Integer agate data_type (#8895)
  • Make relation filtering None-tolerant for maximal flexibility across adapters. (#8974)
  • Update run_results.json from previous versions of dbt to support deferral and rerun from failure (#9010)
  • Use MANIFEST.in to recursively include all jinja templates; fixes issue where some templates were not included in the distribution (#9016)
  • Fix git repository with subdirectory for Deps (#9000)
  • Use seed file from disk for unit testing if rows not specified in YAML config (#8652)
  • Fix formatting of tarball information in packages-lock.yml (#9062)
  • deps: Lock git packages to commit SHA during resolution (#9050)
  • deps: Use PackageRenderer to read package-lock.json (#9127)
  • Ensure we produce valid jsonschema schemas for manifest, catalog, run-results, and sources (#8991)
  • Get sources working again in dbt docs generate (#9119)
  • Fix parsing f-strings in python models (#6976)
  • Preserve the value of vars and the --full-refresh flags when using retry. (#9112)
  • fix lock-file bad indentation (#9319)
  • fix configuration of turning test warnings into failures with WARN_ERROR_OPTIONS (#7761)
  • Support reasonably long unit test names (#9015)
  • Fix back-compat parsing for model-level 'tests', source table-level 'tests', and 'tests' defined on model versions (#9411)
  • Fix retry command run from CLI (#9444)
  • Fix seed and source selection in dbt docs generate (#9161)
  • Add TestGenerateCatalogWithExternalNodes, include empty nodes in node selection during docs generate (#9456)
  • Fix node type plurals in FoundStats log message (#9464)
  • Run manifest upgrade preprocessing on any older manifest version, including v11 (#9487)
  • Update 'compiled_code' context member logic to route based on command ('clone' or not). Reimplement 'sql' context member as wrapper of 'compiled_code'. (#9502)
  • Fix bug where Semantic Layer filter strings are parsed into lists. (#9507)
  • Initialize invocation context before test fixtures are built. (##9489)
  • Fix conflict with newer versions of Snowplow tracker (#8719)
  • When patching versioned models, set constraints after config (#9364)
  • only include unmodified semantic mdodels in state:modified selection (#9548)
  • Set query headers when manifest is passed in to dbtRunner (#9546)
  • Store node_info in node associated logging events (#9557)
  • Fix Semantic Model Compare node relations (#9548)
  • Tighten exception handling to avoid worker thread hangs. (#9583)
  • Clearer no-op logging in stubbed SavedQueryRunner (#9533)
  • Fix node_info contextvar handling so incorrect node_info doesn't persist (#8866)
  • Add target-path to retry (#8948)
  • Do not add duplicate input_measures (#9360)
  • Throw a ParsingError if a primary key constraint is defined on multiple columns or at both the column and model level. (#9581)
  • Bug fix: don't parse Jinja in filters for input metrics or measures. (#9582)
  • Fix traceback parsing for exceptions raised due to csv fixtures moved into or out of fixture/subfolders. (#9570)
  • Fix partial parsing KeyError on deleted schema files (#8860)
  • Support saved queries in dbt list (#9532)
  • include sources in catalog.json when over 100 relations selected for catalog generation (#9755)
  • Support overriding macros in packages in unit testing (#9624)
  • Handle exceptions for failing on-run-* hooks in source freshness (#9511)
  • Validation of unit test parsing for incremental models (#9593)
  • Fix use of retry command on command using defer (#9770)
  • Make args variable to be un-modified by dbt.invoke(args) (#8938, #9787)
  • Only create the packages-install-path / dbt_packages folder during dbt deps (#6985, #9584)
  • Unit test path outputs (#9608)
  • Fix assorted source freshness edgecases so check is run or actionable information is given (#9078)
  • "Fix Docker release process to account for both historical and current versions of dbt-postgres (#9827)
  • Exclude password-like fields for considering reparse (#9795)
  • Fixed query comments test (#9860)
  • Begin warning people about spaces in model names (#9397)
  • Add NodeRelation to SavedQuery Export (#9534)
  • Disambiguiate FreshnessConfigProblem error message (#9891)
  • Use consistent secret scrubbing with the log function. (#9987)
  • Validate against empty strings in package definitions (#9985)
  • Fix default value for indirect selection in selector cannot overwritten by CLI flag and env var (#9976, #7673)
  • Simplify error message if test severity isn't 'warn' or 'error' (#9715)
  • Support overriding source level loaded_at_field with a null table level definition (#9320)
  • Undo conditional agate import to prevent UnresolvedTypeReferenceError during RunResult serialization (#10098)
  • Restore previous behavior for --favor-state: only favor defer_relation if not selected in current command" (#10107)
  • Unit test fixture (csv) returns null for empty value (#9881)

Docs

Under the Hood

  • Added more type annotations. (#8537)
  • Add unit testing functional tests (#8512)
  • Remove usage of dbt.include.global_project in dbt/adapters (#8925)
  • Add a no-op runner for Saved Qeury (#8893)
  • remove dbt.flags.MP_CONTEXT usage in dbt/adapters (#8967)
  • Remove usage of dbt.flags.LOG_CACHE_EVENTS in dbt/adapters (#8969)
  • Move CatalogRelationTypes test case to the shared test suite to be reused by adapter maintainers (#8952)
  • Treat SystemExit as an interrupt if raised during node execution. (#n/a)
  • Removing unused 'documentable' (#8871)
  • Remove use of dbt/core exceptions in dbt/adapter (#8920)
  • Cache dbt plugin modules to improve integration test performance (#9029)
  • Consolidate deferral methods & flags (#7965, #8715)
  • Fix test_current_timestamp_matches_utc test; allow for MacOS runner system clock variance (#9057)
  • Remove usage of dbt.deprecations in dbt/adapters, enable core & adapter-specific event types and protos (#8927, #8918)
  • Clean up unused adaptor folders (#9123)
  • Move column constraints into common/contracts, removing another dependency of adapters on core. (#9024)
  • Move dbt.semver to dbt.common.semver and update references. (#9039)
  • Move lowercase utils method to common (#9180)
  • Remove usages of dbt.clients.jinja in dbt/adapters (#9205)
  • Remove usage of dbt.contracts in dbt/adapters (#9208)
  • Remove usage of dbt.contracts.graph.nodes.ResultNode in dbt/adapters (#9214)
  • Introduce RelationConfig Protocol, consolidate Relation.create_from (#9215)
  • remove manifest from adapter.set_relations_cache signature (#9217)
  • remove manifest from adapter catalog method signatures (#9218)
  • Move BaseConfig, Metadata and various other contract classes from model_config to common/contracts/config (#8919)
  • Add MacroResolverProtocol, remove lazy loading of manifest in adapter.execute_macro (#9244)
  • pass query header context to MacroQueryStringSetter (#9249, #9250)
  • add macro_context_generator on adapter (#9247)
  • pass mp_context to adapter factory as argument instead of import (#9025)
  • have dbt-postgres use RelationConfig protocol for materialized views' (#9292)
  • move system.py to common as dbt-bigquery relies on it to call gcloud (#9293)
  • Reorganizing event definitions to define core events in dbt/events rather than dbt/common (#9152)
  • move exceptions used only in dbt/common to dbt/common/exceptions (#9332)
  • Remove usage of dbt.adapters.factory in dbt/common (#9334)
  • Accept valid_error_names in WarnErrorOptions constructor, remove global usage of event modules (#9337)
  • Move result objects to dbt.artifacts (#9193)
  • dbt Labs OSS standardization of docs and templates. (#9252)
  • Add dbt-common as a dependency and remove dbt/common (#9357)
  • move cache exceptions to dbt/adapters (#9362)
  • Clean up macro contexts. (#9422)
  • Add the @requires.manifest decorator to the retry command. (#9426)
  • Move WritableManifest + Documentation to dbt/artifacts (#9378, #9379)
  • Define Macro and Group resources in dbt/artifacts (#9381, #9382)
  • Move SavedQuery data definition to dbt/artifacts (#9386)
  • Migrate data parts of Metric node to dbt/artifacts (#9383)
  • Move data portion of SemanticModel to dbt/artifacts (#9387)
  • Move data parts of Exposure class to dbt/artifacts (#9380)
  • Split up deferral across parsing (adding 'defer_relation' from state manifest) and runtime ref resolution" (#9199)
  • Start using Mergeable from dbt-common (#9505)
  • Move manifest nodes to artifacts (#9388)
  • Move data parts of SourceDefinition class to dbt/artifacts (#9384)
  • Remove uses of Replaceable class (#7802)
  • Make dbt-core compatible with Python 3.12 (#9007)
  • Restrict protobuf to major version 4. (#9566)
  • Remove references to dbt.tracking and dbt.flags from dbt/artifacts (#9390)
  • Remove unused key wildcard from MethodName enum (#9641)
  • Implement primary key inference for model nodes (#9652)
  • Define UnitTestDefinition resource in dbt/artifacts/resources (#9667)
  • Use Manifest instead of WritableManifest in PreviousState and _get_deferred_manifest (#9567)
  • Improve dbt CLI speed (#4627)
  • Include node_info in various Result events (#9619)
  • Remove non dbt.artifacts dbt.* imports from dbt/artifacts (#9926)
  • Migrate to using error_tag provided by dbt-common (#9914)
  • Add a test for semantic manifest and move test fixtures needed for it (#9665)
  • Raise deprecation warning if installed package overrides built-in materialization (#9971)
  • Use the SECRET_ENV_PREFIX from dbt_common instead of duplicating it in dbt-core (#10018)
  • Enable use of record mode via environment variable (#10045)
  • Consistent naming + deprecation warnings for "legacy behavior" flags (#10062)
  • Enable use of context in serialization (#10093)

Dependencies

  • Bump actions/checkout from 3 to 4 (#8781)
  • Begin using DSI 0.4.x (#8892)
  • Update typing-extensions version to >=4.4 (#9012)
  • Bump ddtrace from 2.1.7 to 2.3.0 (#9132)
  • Bump freezegun from 0.3.12 to 1.3.0 (#9197)
  • Bump actions/setup-python from 4 to 5 (#9267)
  • Bump actions/download-artifact from 3 to 4 (#9374)
  • Relax pathspec upper bound version restriction (#9373)
  • remove dbt/adapters and add dependency on dbt-adapters (#9430)
  • Bump actions/upload-artifact from 3 to 4 (#9470)
  • Bump actions/cache from 3 to 4 (#9471)
  • Bump peter-evans/create-pull-request from 5 to 6 (#9552)
  • Restrict protobuf to 4.* versions (#9566)
  • Bump codecov/codecov-action from 3 to 4 (#9659)
  • Cap dbt-semantic-interfaces version range to <0.6 (#9671)
  • Bump python from 3.10.7-slim-nullseye to 3.11.2-slim-bullseye in /docker (#9687)
  • bump dbt-common to accept major version 1 (#9690)
  • Remove duplicate dependency of protobuf in dev-requirements (#9830)
  • Bump black from 23.3.0 to >=24.3.0,<25.0 (#8074)
  • Update the agate pin to "agate>=1.7.0,<1.10" (#9934)

Security

Contributors

Check out latest releases or
releases around dbt-core 1.8.0

Don't miss a new dbt-core release

NewReleases is sending notifications on new releases.

Get notifications