dbt 0.15.0 (November 25, 2019)
Breaking changes
- Support for Python 2.x has been dropped as it will no longer be supported on January 1, 2020
- Compilation errors in .yml files are now treated as errors instead of warnings (#1493, #1751)
- The 'table_name' field field has been removed from Relations
- The existing
compile
andexecute
rpc tasks have been renamed tocompile_sql
andexecute_sql
(#1779, #1798) (docs) - Custom materializations must now manage dbt's Relation cache (docs)
Installation notes:
dbt v0.15.0 uses the psycopg2-binary
dependency (instead of psycopg2
) to simplify installation on platforms that do not have a compiler toolchain installed. pip users should run pip uninstall psycopg2
before upgrading their version of dbt to 0.15.0. If after upgrading you experience segmentation faults, crashes, or installation errors, you can set the DBT_PSYCOPG2_NAME
environment variable to psycopg2
to change the dependency that dbt installs. This may require a compiler toolchain and development libraries.
$ DBT_PSYCOPG2_NAME=psycopg2 pip install dbt
You may also install specific dbt plugins directly by name. This has the advantage of only installing the Python requirements needed for your particular database:
# Installing dbt-snowflake and dbt-bigquery directly bypasses the psycopg2 dependency
$ pip install dbt-snowflake
$ pip install dbt-bigquery
# Installing dbt-postgres and dbt-redshift are also supported
$ pip install dbt-postgres
$ pip install dbt-redshift
Changes
Core
Features
- Add a JSON logger (#1237, #1791) (docs)
- Add structured logging to dbt (#1704, #1799, #1715, #1806)
- Add partial parsing option to the profiles.yml file (#1835, #1836, #1487) (docs)
- Support configurable query comments in SQL queries (#1643, #1864) (docs)
- Support atomic full-refreshes for incremental models (#525, #1682)
- Support snapshot configs in dbt_project.yml (#1613, #1759) (docs)
- Support cache modifications in materializations (#1683, #1770) (docs)
- Support
quote
parameter to Accepted Values schema tests (#1873, #1876) (docs) - Support Python 3.8 (#1886)
- Support filters in sources for
dbt source snapshot-freshness
invocation (#1495, #1776) (docs) - Support external table configuration in yml source specifications (#1784)
- Improve CLI output when running snapshots (#1768, #1769)
Fixes
- Fix for unhelpful error message for malformed source/ref inputs (#1660, #1809)
- Fix for lingering backup tables when incremental models are full-refreshed (#1933, #1931)
- Fix for confusing error message when errors are encountered during compilation (#1807, #1839)
- Fix for logic error affecting the two-argument flavor of the
ref
function (#1504, #1515) - Fix for invalid reference to dbt.exceptions (#1569, #1609)
- Fix for "cannot run empty query" error when pre/post-hooks are empty (#1108, #1719)
- Fix for confusing error when project names shadow context attributes (#1696, #1748)
- Fix for incorrect database logic in docs generation which resulted in columns being "merged" together across tables (#1708, #1774)
- Fix for seed errors located in dependency packages (#1723, #1723)
- Fix for confusing error when schema tests return unexpected results (#1808, #1903)
- Fix for twice-compiled
statement
block contents (#1717, #1719) - Fix for inaccurate output in
dbt run-operation --help
(#1767, #1777) - Fix for file rotation issues concerning the
logs/dbt.log
file (#1863, #1865, #1871) - Fix for missing quotes in incremental model build queries (#1847, #1888)
- Fix for incorrect log level in
printer.print_run_result_error
(#1818, #1823)
Docs
- Show seeds and snapshots in the Project and Database views (docs#37, docs#25, docs#52)
- Show sources in the Database tree view (docs#20, docs#52)
- Show edges in the DAG between models and seeds (docs#15, docs#52)
- Show Accepted Values tests and custom schema tests in the column list for models (docs#52)
- Fix links for "Refocus on node" and "View documentation" in DAG context menu for seeds (docs#52)
Server
- Support docs generation (#1781, #1801)
- Support custom tags (#1822, #1828)
- Support invoking
deps
on the rpc server (#1834, #1837) - Support invoking
run-operation
andsnapshot
on the rpc server (#1875, #1878) - Suppport
--threads
argument tocli_args
method (#1897, #1909) - Support reloading the manifest when a SIGHUP signal is received (#1684, #1699)
- Support invoking
compile
,run
,test
, andseed
on the rpc server (#1488, #1652) - Support returning compilation logs from the last compile in the
status
method (#1703, #1775) - Support asyncronous
compile_sql
andrun_sql
methods (#1706, #1735) - Improve re-compilation performance (#1824, #1830)
Postgres / Redshift
- Support running dbt against schemas which contain materialized views on Postgres (#1698, #1833)
- Support distyle AUTO in Redshift model configs (#1882, #1885) (docs)
- Fix for internal errors when run against mixed-case logical databases (#1800, #1936)
Snowflake
- Support
copy grants
option in Snowflake model configs (#1744, #1747) (docs) - Support warehouse configuration in Snowflake model configs (#1358, #1899, #1788, #1901) (docs)
- Support secure views in Snowflake model configs (#1730, #1743) (docs)
- Fix for unclosed connections preventing dbt from exiting when Snowflake is used with client_session_keep_alive (#1271, #1749)
- Fix for errors on Snowflake when dbt schemas contain
LOCAL TEMPORARY
tables (#1869, #1872)
BigQuery
- Support KMS Encryption in BigQuery model configs (#1829, #1851) (docs)
- Improve docs generation speed by leveraging the information schema (#1576, #1795)
- Fix for cache errors on BigQuery when dataset names are capitalized (#1810, #1881)
- Fix for invalid query generation when multiple
options
are provided to acreate table|view
query (#1786, #1787) - Use
client.delete_dataset
to drop BigQuery datasets atomically (#1887, #1881)
Under the Hood
Dependencies
- Drop support for
networkx 1.x
(#1577, #1814) - Upgrade
werkzeug
to 0.15.6 (#1697, #1814) - Pin
psycopg2
dependency to 2.8.x to prevent segfaults (#1221, #1898) - Set a strict upper bound for
jsonschema
dependency (#1817, #1821, #1932)
Everything else
- Provide test names and kwargs in the manifest (#1154, #1816)
- Replace JSON Schemas with data classes (#1447, #1589)
- Include test name and kwargs in test nodes in the manifest (#1154, #1816)
- Remove logic around handling
archive
blocks in thedbt_project.yml
file (#1580, #1581) - Remove the APIObject class (#1762, #1780)
Contributors
Thanks all for your contributions to dbt! 🎉
- @captainEli (#1809)
- @clausherther (#1876)
- @jtcohen6 (#1784)
- @tbescherer (#1515)
- @aminamos (#1609)
- @JusLarsen (#1903)
- @heisencoder (#1823)
- @tjengel (#1885)
- @Carolus-Holman (#1747, #1743)
- @kconvey (#1851)
- @darrenhaken (#1787)
Louisa May Alcott
Louisa May Alcott was born in 1832 in Philadelphia, Pennsylvania. Alcott's early education included lessons from noted transcendentalists Henry David Thoreau, Ralph Waldo Emerson, Nathaniel Hawthorne, Margaret Fuller, and Julia Ward Howe, all of whom were family friends. In 1868, Alcott wrote Little Women, a children's novel set in the Alcott family home some miles due north of Philadelphia. The novel is loosely based on Alcott's childhood experiences with her three sisters. The novel was well-received at the time and is still popular today.
As an adult, Alcott was an abolitionist and a feminist. In 1847, she and her family served as station masters on the Underground Railroad. During this time, they housed a fugitive slave and had discussions with Frederick Douglass, a national leader of the abolitionist movement in America. Alcott read and admired the Declaration of Sentiments, published by the Seneca Falls Convention on women's rights, advocating for women's suffrage.
To learn more about Louisa May Alcott's life, consider visiting her Boston home (featured on the Boston Women's Heritage Trail) or her childhood home, Orchard House, which is now a museum that pays homage to Louisa May Alcott and her family with a focus on education.