dbt 0.3.0

dbt version 0.3.0

on Python PyPI

dbt v0.3.0

Version 0.3.0 comes with the following updates:

1. Parallel model creation #83

dbt will analyze the model dependency graph and can create models in parallel if possible. In practice, this can significantly speed up the amount of time it takes to complete dbt run. The number of threads dbt uses must be between 1 and 8. To configure the number of threads dbt uses, add the threads key to your dbt target in ~/.dbt/profiles.yml, eg:

user:
  outputs:
    my-redshift:
      type: redshift
      threads: 4         # execute up to 4 models concurrently
      host: localhost
      ...
  run-target: my-redshift

For a complete example, check out a sample profiles.yml file

2. Fail only within a single dependency chain #63

If a model cannot be created, it won't crash the entire dbt run process. The errant model will fail and all of its descendants will be "skipped". Other models which do not depend on the failing model (or its descendants) will still be created.

3. Logging #64, #65

dbt will log output from the dbt run and dbt test commands to a configurable logging directory. By default, this directory is called logs/. The log filename is dbt.log and it is rotated on a daily basic. Logs are kept for 7 days.

To change the name of the logging directory, add the following line to your dbt_project.yml file:

log-path: "my-logging-directory" # will write logs to my-logging-directory/dbt.log

4. Minimize time models are unavailable in the database #68

Previously, dbt would create models by:

1. dropping the existing model
2. creating the new model

This resulted in a significant amount of time in which the model was inaccessible to the outside world. Now, dbt creates models by:

1. creating a temporary model {model-name}__dbt_tmp
2. dropping the existing model
3. renaming the tmp model name to the actual model name

5. Arbitrarily deep nesting #50

Previously, all models had to be located in a directory matching models/{model group}/{model_name}.sql. Now, these models can be nested arbitrarily deeply within a given dbt project. For instance, models/snowplow/sessions/transformed/transformed_sessions.sql is a totally valid model location with this release.

To configure these deeply-nested models, just nest the config options within the dbt_project.yml file. The only caveat is that you need to specify the dbt project name as the first key under the models object, ie:

models:
  'Your Project Name':
    snowplow:
      sessions:
        transformed:
          transformed_sessions:
            enabled: true

More information is available on the issue and in the sample dbt_project.yml file

6. don't try to create a schema if it already exists #66

dbt run would execute create schema if not exists {schema}. This would fail if the dbt user didn't have sufficient permissions to create the schema, even if the schema already existed! Now, dbt checks for the schema existence and only attempts to create the schema if it doesn't already exist.

7. Semantic Versioning

The previous release of dbt was v0.2.3.0 which isn't a semantic version. This and all future dbt releases will conform to semantic version in the format {major}.{minor}.{patch}.