sqlfluff/sqlfluff 2.3.3

[2.3.3] - 2023-10-13

on GitHub

Highlights

There's a lot in this release. Most of it is under the covers and so shouldn't cause any breaking changes for most users. If your use case depends on some of the internals of SQLFluff, you may find some breaking changes. The bigger changes are:

• Python 3.12 support is now official (although older releases may also work as only a few changes were required for full 3.12 support).
• We've done a significant re-write of the parsing engine to remove some unnecessary segment manipulation and get us closer to "single pass" parsing. This changes the internal API being used on any .match() methods, and also removes the parse_grammar attribute on any dialect segments. We are not aware of any 3rd party libraries which rely on these APIs however and so have not triggered a more major release. These lead to significant performance improvements during parsing.
• Standardisation of terminators in the parser, and the introduction of the ParseMode option has enabled the removal of the StartsWith, GreedyUntil and EphemeralSegment parser classes.
• Several validation checks have been revised in this release, which should both improve performance (by reducing duplication), but also be more effective in preventing the application of any fixes which would result in unparsable files.

Alongside the big things this also includes a host of bugfixes, dialect improvements and CI/testing improvements.

This release also sees a bumper crop of new contributors, thanks to @dehume, @andychannery, @Kylea650, @robin-alphasophia, @jtbg, @r-petit, @bpfaust, @freewaydev & @abdel who all made the first contibutions in this release! 🎉🎉🎉

What’s Changed

• Oracle space between alias and column reference #5313 @joaostorrer
• Don't apply LT05 on templated rebreak locations #5096 #5318 @alanmcruickshank
• Disable JJ01 unless jinja active #5319 @alanmcruickshank
• Cache the BaseSegment hash in reflow #5320 @alanmcruickshank
• Better error reporting for invalid macros #5317 @alanmcruickshank
• Add support for begin atomic functions in Postgres #5316 @joaostorrer
• Fix parsing when statement uses plus_sign_join and function in Oracle #5315 @joaostorrer
• Update rule docs with correct config #5314 @alanmcruickshank
• Resolve #5258. More robust algorithm for multiline fix. #5309 @alanmcruickshank
• BigQuery: Add support for BEGIN..EXCEPTION...END block #5307 @abdel
• Refine placement of metas around templated blocks #5294 @alanmcruickshank
• Extend ruff checking to docstring rules #5302 @alanmcruickshank
• Fix for strange TSQL bugs #5306 @alanmcruickshank
• Staging PR for #5282 #5305 @alanmcruickshank
• Resolve some strange whitespace indentation behaviour #5292 @alanmcruickshank
• Simplify _process_lint_result #5304 @alanmcruickshank
• Performance improvement on segment comparison #5303 @alanmcruickshank
• Refactor LT09 #5299 @alanmcruickshank
• Change drop function to allow DropBehaviourGrammar with space after function name #5295 @joaostorrer
• Resolve click import options on autocomplete #5293 @alanmcruickshank
• Updated docstrings with missing args/returns/etc info, added missing docstrings, minor formatting fixes. #5278 @freewaydev
• Use ruff rule I replace isort #5289 @zhongjiajie
• Snowflake: Parse ALTER DATABASE statement #5284 @jmks
• Snowflake: Parse ALTER ACCOUNT statements #5283 @jmks
• Snowflake: create AlterProcedureStatementSegment #5291 @moreaupascal56
• Rewrite of matching interface #5230 @alanmcruickshank
• Follow noqa in block comments #5133 @daviewales
• Fix insert on conflict with function in Postgres #5286 @joaostorrer
• Add support for Pivot and Unpivot clauses in Oracle #5285 @joaostorrer
• Adding "create table as" for greenplum dialect #5173 @bpfaust
• Update CI to python 3.12 #5267 @alanmcruickshank
• Snowflake: Add CreateResourceMonitorStatementSegment & AlterResourceMonitorStatementSegment #5272 @moreaupascal56
• [TSQL] Add create fulltext index statement segment class #5274 @r-petit
• Snowflake: Add CreateSequenceStatementSegment & AlterSequenceStatementSegment #5270 @moreaupascal56
• Add CommaSegment to AlterWarehouseStatementSegment SET clause #5268 @moreaupascal56
• Snowflake: Parse EXECUTE IMMEDIATE clause #5275 @jmks
• TSQL: Add missing HISTORY_RETENTION_PERIOD sequence to the table option segment #5273 @r-petit
• Snowflake: Fix ScalingPolicy and WarehouseType Refs in WarehouseObjectProperties and use ObjectReferenceSegment in AlterWarehouseStatementSegment #5264 @moreaupascal56
• Finish the removal of GreedyUntil #5263 @alanmcruickshank
• Add support for date operations with intervals in Oracle #5262 @joaostorrer
• Change RawSegment type to instance_types #5253 @alanmcruickshank
• Revise MatchableType -> Matchable #5252 @alanmcruickshank
• Unnest fixing and re-address validation triggers #5249 @alanmcruickshank
• Resolves #5174: Snowflake alter table constraint #5247 @andychannery
• Bring together the generic segments #5243 @alanmcruickshank
• minor: update docs with correct link to airflow ds_filter #5244 @jtbg
• #5245 - Snowflake dialect: Adds support for variable definitions in scripting blocks #5246 @robin-alphasophia
• Introduce "word" segment #5234 @alanmcruickshank
• #5239 Added (basic) support for properly linted Snowflake scripting #5242 @robin-alphasophia
• Allow Snowflake pipe integration to be a quoted or unquoted #5241 @Kylea650
• Fix LT01 alignment regression #4023 #5238 @alanmcruickshank
• Add support for oracle non ansi joins #5231 @joaostorrer
• add azure_storage_queue and quoted providers #5236 @Kylea650
• Set type automatically within the lexer. #5232 @alanmcruickshank
• Resolve #5225: Snowflake unparsable select replace #5227 @andychannery
• Spark Accessor Grammars #5226 @alanmcruickshank
• Test Script Timing #5228 @alanmcruickshank
• Unify lexer names and types for brackets #5229 @alanmcruickshank
• Resolve #3176: Snowflake unparsable array casting #5224 @andychannery
• BigQuery system time syntax #5220 @greg-finley
• Parser test nits #5217 @alanmcruickshank
• Remove parse_grammar #5189 @alanmcruickshank
• Revise segment whitespace validation #5194 @alanmcruickshank
• Add support for bpchar datatype in postgres #5215 @joaostorrer
• Resolve #5203: BaseSegment.copy() isolation #5206 @alanmcruickshank
• Update Materialize Syntax #5210 @dehume
• Validate fix parsing based on match_grammar #5196 @alanmcruickshank
• Position assertions in BaseSegment #5209 @alanmcruickshank
• MySQL bracketed column constraint #5208 @greg-finley
• Dialect spacing & quoting issues #5205 @alanmcruickshank
• Update comment workflow again #5201 @alanmcruickshank
• Stash PR Number and hydrate later #5200 @alanmcruickshank
• Fix API issues in github comment action #5199 @alanmcruickshank
• Consistency check in root parse #5191 @alanmcruickshank
• PR Comment action #5192 @alanmcruickshank
• Cache python dependencies in GHA #5193 @alanmcruickshank
• Add support for create server, create user mapping and import foreign schema in postgres #5185 @joaostorrer
• Terminators on Anything() + Strip most of the other parse_grammar #5186 @alanmcruickshank
• Parse modes for AnyNumberOf #5187 @alanmcruickshank
• Introduce parse_mode and remove StartsWith & EphemeralSegment. #5167 @alanmcruickshank

New Contributors

• @dehume made their first contribution in #5210
• @andychannery made their first contribution in #5224
• @Kylea650 made their first contribution in #5236
• @robin-alphasophia made their first contribution in #5242
• @jtbg made their first contribution in #5244
• @r-petit made their first contribution in #5273
• @bpfaust made their first contribution in #5173
• @freewaydev made their first contribution in #5278
• @abdel made their first contribution in #5307