sqlfluff/sqlfluff 1.3.0

on GitHub

Highlights

This release brings several potentially breaking changes to the underlying parse tree. For users of the cli tool in a linting context you should notice no change. If however your application relies on the structure of the SQLFluff parse tree or the naming of certain elements within the yaml format, then this may not be a drop-in replacement. Specifically:

• The addition of a new end_of_file meta segment at the end of the parse structure.
• The addition of a template_loop meta segment to signify a jump backward in the source file within a loop structure (e.g. a jinja for loop).
• Much more specific types on some raw segments, in particular identifier and literal type segments will now appear in the parse tree with their more specific type (which used to be called name) e.g. naked_identifier, quoted_identifier, numeric_literal etc...

If using the python api, the parent type (such as identifier) will still register if you call .is_type("identifier"), as this function checks all inherited types. However the eventual type returned by .get_type() will now be (in most cases) what used to be accessible at .name. The name attribute will be deprecated in a future release.

Other highlights:

• New command-line option --show-lint-violations to show details on unfixable errors when running sqlfluff fix.
• Improved consistency of process exit codes.
• Short CLI options for many common options.
• Jinja templater: When --ignore=templating is enabled, undefined Jinja variables now take on "reasonable" default values rather than blank string (""). This can streamline initial rollout of SQLFluff by reducing or eliminating the need to configure templater variables.

There are also a ton of other features and bug fixes in this release, including first-time contributions from 11 new contributors! 🎉

What’s Changed

• T-SQL: ALTER TABLE DROP COLUMN #3749 @greg-finley
• Add "# pragma: no cover" to work around sporadic, spurious coverage failure #3767 @barrywhart
• Add end_of_file and template_loop markers #3766 @alanmcruickshank
• Provide usage examples for new users #3765 @sirlark
• SQLite: deferrable in create table statement #3757 @RossOkuno
• When ignore=templating and fix_even_unparsable=True, provide defaults for missing vars #3753 @barrywhart
• BigQuery: Support Materialized Views #3759 @yoichi
• Enhance L062 to ignore blocked words in comments #3754 @tunetheweb
• Fix bug where undefined Jinja variable in macro file crashes linter #3751 @barrywhart
• Migrate analysis, functional and testing to utils #3743 @alanmcruickshank
• Build out rule crawling mechanisms #3717 @alanmcruickshank
• Add current_timestamp to Redshift as a bare function #3741 @RossOkuno
• BigQuery: Fix parsing parameterized data types #3735 @yoichi
• Change MySQL Create Statement Equals Segment to Optional #3730 @keyem4251
• SQLite: add parsing of INSERT statement #3734 @imrehg
• SPARKSQL: Support Delta Lake Drop Column Clause in ALTER TABLE #3727 @R7L208
• Add short versions of several cli options #3732 @alanmcruickshank
• Build out type hints in Grammars #3718 @alanmcruickshank
• dbt 1.3.0 compatibility #3708 @edgarrmondragon
• Revise no cover direction and remove unused code. #3723 @alanmcruickshank
• Update broken flattr link #3720 @alanmcruickshank
• BigQuery: remove key from unreserved keywords list #3719 @sabrikaragonen
• Bigquery reset primary and foreign keys #3714 @sabrikaragonen
• Name Deprecation (Part 1) #3701 @alanmcruickshank
• Teradata: Add two TdTableConstraints #3690 @greg-finley
• Redshift: support expressions in array accessors #3706 @chronitis
• Handle logging issues at teardown #3703 @alanmcruickshank
• L028, L032: Fix bug where fixes were copying templated table names #3699 @barrywhart
• L042: Autofix sometimes results in "fix looping", hitting the linter "loop limit" #3697 @barrywhart
• L042: Address corner cases where fix corrupts the SQL #3694 @barrywhart
• T-SQL: Properly parse collation names #3686 @borchero
• Allow escaping single quotes in single-quoted literal with '' #3682 @pdebelak
• T-SQL: Fix indentation after JOIN/APPLY clauses with no ON statement #3684 @borchero
• T-SQL: Parse DATEPART date type as date type instead of column name #3681 @borchero
• T-SQL: Allow COLLATE clause in JOIN conditions #3680 @borchero
• T-SQL: Fix parsing of CREATE VIEW statements with column name syntax #3669 @borchero
• Fix typo in github issue template #3674 @alanmcruickshank
• Add Athena issue label #3676 @greg-finley
• Set issue dialect labels via Github Actions #3666 @greg-finley
• Allow configuration of processes from config #3662 @alanmcruickshank
• Reposition before recursion in fixes to avoid internal error #3658 @alanmcruickshank
• Use UUIDs for matching #3661 @alanmcruickshank
• Postgres: Add dialect-specific bare functions #3660 @WittierDinosaur
• Postgres: Add CALL Support #3659 @WittierDinosaur
• ANSI - Add support for INTERSECT ALL and EXCEPT ALL #3657 @WittierDinosaur
• Option to show errors on fix #3610 @chaimt
• L042: Fix internal error "Attempted to make a parent marker from multiple files" #3655 @barrywhart
• L026: Add support for merge_statement #3654 @barrywhart
• Add handling for Redshift CONVERT function data type argument #3653 @pdebelak
• Deduplicate files before and during templating #3629 @alanmcruickshank
• Rationalise Rule Imports #3631 @alanmcruickshank
• Handle Jinja {% call ... %} blocks #3648 @barrywhart
• SPARKSQL: Add Delta Lake Constraints syntax to ALTER TABLE #3643 @R7L208
• Redshift: syntax for array unnesting with index #3646 @chronitis
• Snowflake - ALTER TABLE IF EXISTS and WHEN SYSTEM$STREAM_HAS_DATA() #3641 @chrisalexeev
• L057: In BigQuery, allow hyphens by default #3645 @barrywhart
• Better messages for partial indentation in L003 #3634 @pdebelak
• Add INTEGER to PrimitiveTypeSegment for Sparksql #3624 @ciwassano
• Bump version in gettingstarted.rst via the release script #3642 @greg-finley
• Improve handling of BigQuery hyphenated table names #3638 @barrywhart
• update sqlfluff version in gettingstareted.rst #3639 @keyem4251
• L016: Ignore jinja comments if ignore_comment_clauses=True #3637 @barrywhart
• Add errors for redundant definitions. #3626 @alanmcruickshank
• Object Literals #3620 @alanmcruickshank
• Dialect Crumbs #3625 @alanmcruickshank
• Consistent return codes #3608 @alanmcruickshank

New Contributors

• @keyem4251 made their first contribution in #3639
• @ciwassano made their first contribution in #3624
• @chronitis made their first contribution in #3646
• @chaimt made their first contribution in #3610
• @borchero made their first contribution in #3669
• @sabrikaragonen made their first contribution in #3714
• @edgarrmondragon made their first contribution in #3708
• @imrehg made their first contribution in #3734
• @yoichi made their first contribution in #3735
• @RossOkuno made their first contribution in #3741
• @sirlark made their first contribution in #3765