dbt 0.15.0 (November 25, 2019)

Breaking changes

• Support for Python 2.x has been dropped as it will no longer be supported on January 1, 2020
• Compilation errors in .yml files are now treated as errors instead of warnings (#1493, #1751)
• The 'table_name' field field has been removed from Relations
• The existing compile and execute rpc tasks have been renamed to compile_sql and execute_sql (#1779, #1798) (docs)
• Custom materializations must now manage dbt's Relation cache (docs)

Installation notes:

dbt v0.15.0 uses the psycopg2-binary dependency (instead of psycopg2) to simplify installation on platforms that do not have a compiler toolchain installed. pip users should run pip uninstall psycopg2 before upgrading their version of dbt to 0.15.0. If after upgrading you experience segmentation faults, crashes, or installation errors, you can set the DBT_PSYCOPG2_NAME environment variable to psycopg2 to change the dependency that dbt installs. This may require a compiler toolchain and development libraries.

$ DBT_PSYCOPG2_NAME=psycopg2 pip install dbt

You may also install specific dbt plugins directly by name. This has the advantage of only installing the Python requirements needed for your particular database:

# Installing dbt-snowflake and dbt-bigquery directly bypasses the psycopg2 dependency
$ pip install dbt-snowflake
$ pip install dbt-bigquery

# Installing dbt-postgres and dbt-redshift are also supported
$ pip install dbt-postgres
$ pip install dbt-redshift

Changes

Core

Features

• Add a JSON logger (#1237, #1791) (docs)
• Add structured logging to dbt (#1704, #1799, #1715, #1806)
• Add partial parsing option to the profiles.yml file (#1835, #1836, #1487) (docs)
• Support configurable query comments in SQL queries (#1643, #1864) (docs)
• Support atomic full-refreshes for incremental models (#525, #1682)
• Support snapshot configs in dbt_project.yml (#1613, #1759) (docs)
• Support cache modifications in materializations (#1683, #1770) (docs)
• Support quote parameter to Accepted Values schema tests (#1873, #1876) (docs)
• Support Python 3.8 (#1886)
• Support filters in sources for dbt source snapshot-freshness invocation (#1495, #1776) (docs)
• Support external table configuration in yml source specifications (#1784)
• Improve CLI output when running snapshots (#1768, #1769)

Fixes

• Fix for unhelpful error message for malformed source/ref inputs (#1660, #1809)
• Fix for lingering backup tables when incremental models are full-refreshed (#1933, #1931)
• Fix for confusing error message when errors are encountered during compilation (#1807, #1839)
• Fix for logic error affecting the two-argument flavor of the ref function (#1504, #1515)
• Fix for invalid reference to dbt.exceptions (#1569, #1609)
• Fix for "cannot run empty query" error when pre/post-hooks are empty (#1108, #1719)
• Fix for confusing error when project names shadow context attributes (#1696, #1748)
• Fix for incorrect database logic in docs generation which resulted in columns being "merged" together across tables (#1708, #1774)
• Fix for seed errors located in dependency packages (#1723, #1723)
• Fix for confusing error when schema tests return unexpected results (#1808, #1903)
• Fix for twice-compiled statement block contents (#1717, #1719)
• Fix for inaccurate output in dbt run-operation --help (#1767, #1777)
• Fix for file rotation issues concerning the logs/dbt.log file (#1863, #1865, #1871)
• Fix for missing quotes in incremental model build queries (#1847, #1888)
• Fix for incorrect log level in printer.print_run_result_error (#1818, #1823)

Docs

• Show seeds and snapshots in the Project and Database views (docs#37, docs#25, docs#52)
• Show sources in the Database tree view (docs#20, docs#52)
• Show edges in the DAG between models and seeds (docs#15, docs#52)
• Show Accepted Values tests and custom schema tests in the column list for models (docs#52)
• Fix links for "Refocus on node" and "View documentation" in DAG context menu for seeds (docs#52)

Server

• Support docs generation (#1781, #1801)
• Support custom tags (#1822, #1828)
• Support invoking deps on the rpc server (#1834, #1837)
• Support invoking run-operation and snapshot on the rpc server (#1875, #1878)
• Suppport --threads argument to cli_args method (#1897, #1909)
• Support reloading the manifest when a SIGHUP signal is received (#1684, #1699)
• Support invoking compile, run, test, and seed on the rpc server (#1488, #1652)
• Support returning compilation logs from the last compile in the status method (#1703, #1775)
• Support asyncronous compile_sql and run_sql methods (#1706, #1735)
• Improve re-compilation performance (#1824, #1830)

Postgres / Redshift

• Support running dbt against schemas which contain materialized views on Postgres (#1698, #1833)
• Support distyle AUTO in Redshift model configs (#1882, #1885) (docs)
• Fix for internal errors when run against mixed-case logical databases (#1800, #1936)

Snowflake

• Support copy grants option in Snowflake model configs (#1744, #1747) (docs)
• Support warehouse configuration in Snowflake model configs (#1358, #1899, #1788, #1901) (docs)
• Support secure views in Snowflake model configs (#1730, #1743) (docs)
• Fix for unclosed connections preventing dbt from exiting when Snowflake is used with client_session_keep_alive (#1271, #1749)
• Fix for errors on Snowflake when dbt schemas contain LOCAL TEMPORARY tables (#1869, #1872)

BigQuery

• Support KMS Encryption in BigQuery model configs (#1829, #1851) (docs)
• Improve docs generation speed by leveraging the information schema (#1576, #1795)
• Fix for cache errors on BigQuery when dataset names are capitalized (#1810, #1881)
• Fix for invalid query generation when multiple options are provided to a create table|view query (#1786, #1787)
• Use client.delete_dataset to drop BigQuery datasets atomically (#1887, #1881)

Under the Hood

Dependencies

• Drop support for networkx 1.x (#1577, #1814)
• Upgrade werkzeug to 0.15.6 (#1697, #1814)
• Pin psycopg2 dependency to 2.8.x to prevent segfaults (#1221, #1898)
• Set a strict upper bound for jsonschema dependency (#1817, #1821, #1932)

Everything else

• Provide test names and kwargs in the manifest (#1154, #1816)
• Replace JSON Schemas with data classes (#1447, #1589)
• Include test name and kwargs in test nodes in the manifest (#1154, #1816)
• Remove logic around handling archive blocks in the dbt_project.yml file (#1580, #1581)
• Remove the APIObject class (#1762, #1780)

Contributors

Thanks all for your contributions to dbt! 🎉

• @captainEli (#1809)
• @clausherther (#1876)
• @jtcohen6 (#1784)
• @tbescherer (#1515)
• @aminamos (#1609)
• @JusLarsen (#1903)
• @heisencoder (#1823)
• @tjengel (#1885)
• @Carolus-Holman (#1747, #1743)
• @kconvey (#1851)
• @darrenhaken (#1787)

Louisa May Alcott

Louisa May Alcott was born in 1832 in Philadelphia, Pennsylvania. Alcott's early education included lessons from noted transcendentalists Henry David Thoreau, Ralph Waldo Emerson, Nathaniel Hawthorne, Margaret Fuller, and Julia Ward Howe, all of whom were family friends. In 1868, Alcott wrote Little Women, a children's novel set in the Alcott family home some miles due north of Philadelphia. The novel is loosely based on Alcott's childhood experiences with her three sisters. The novel was well-received at the time and is still popular today.

As an adult, Alcott was an abolitionist and a feminist. In 1847, she and her family served as station masters on the Underground Railroad. During this time, they housed a fugitive slave and had discussions with Frederick Douglass, a national leader of the abolitionist movement in America. Alcott read and admired the Declaration of Sentiments, published by the Seneca Falls Convention on women's rights, advocating for women's suffrage.

To learn more about Louisa May Alcott's life, consider visiting her Boston home (featured on the Boston Women's Heritage Trail) or her childhood home, Orchard House, which is now a museum that pays homage to Louisa May Alcott and her family with a focus on education.