This is a patch release, containing bug fixes:

• Corrupted secondary indexes after dropping and re-adding a table's primary keys in a different order
• LIKE expressions with an OR returned incorrect results
• Better support for unquoted keyword identifiers in parser
• Made the db_working, db_staged, db_head system variables read-only, deprecate detached head mode
• Various DDL statements now parse correctly
• Hex numbers not evaluating correctly in some expressions
• Fixed various console display bugs during long-running operations like dolt pull

Merged PRs

dolt

• 3013: Apply PK ordinals to secondary index pk tag ordering
  The original bug was found by stabs: #2991
  Refer to the GMS PR with a script test for testing the condition that triggers the bug: dolthub/go-mysql-server#881
  A summary of the bug is: Secondary indexes were caching old pk orderings. Schema changes created an inconsistency between secondary indexes and the primary keys they referred to, causing errors in index lookups. This only impacts DDL statements involving a new pk ordering and preexisting secondary indexes.
  There are several layers of indirection between pk source row and secondary index row. When creating an index row, a higher layer of code reads and formats keys in the correct index ordering (for example, reading schema order (x,y,z) -> (y,z,x)). However, we deconstruct and rebuild that row before inserting values into a secondary index. The second constructor drops the ordering in favor of a hashmap of tags, and uses an index embedded tag ordering pks to recreate the row.
  This fix extends schema.SetPkOrdinals to override the preexisting ordering of pks with an indexCollection.SetPks method.
• 3007: go/libraries/doltcore/sqle/writer: Working Set refactor
• 3002: Andy/temp table refactor
  redo of #2993
• 3001: libraries/doltcore/dtestutils/testcommands: fix dropped errors
  This fixes several err variables that were set but then unhandled, as well as an os.Chdir() that looked a little risky without one.
• 2996: Add ORM Bats testing utilities
  This PR setups the basic bats setup for ORMs. Once we have an ORM passing end to end we can expose this to the CI with the relevant workflows.
• 2994: Change transactions per second to sql_transactions_per_second
  Fixes the column name of the result set for a particular benchmarking query.
• 2986: Update queries to include tps as well as more sophisticated percentil…
  This pr fixes some of the queries produced by the TPCC automation. It also adds a query to return TPS
• 2967: Add Ephemeral Printer
  EphemeralPrinter is tool that you can use to print temporary line(s) to the console. Every time Display is called, the previously written lines are cleared, and the new lines are flushed to the output.
  Fixes #2964
• 2966: go/libraries/doltcore/sqle: Remove "Detached Head" mode
  This PR removes support for "detached-head" mode in the SQL context and modifies surrounding functionality to restrict state changes in the SQL context. The central change here is that all SQL writes must be made against a working set and all writes must be made transactionally. With the exception of read-only sessions, all SQL sessions will have an associated working set.
  Currently, SQL sessions can modify their state through special session variables:
  • myDatabase_head_ref: essentially a branch change, changes the current head and also switches to the associated working set.
  • myDatabase_head: similar to the _head_ref variable, but more general in that any commit can be used as a new head. Setting this variable puts the session into detached-head mode. Writes to this session are not transactionally written to a working set. Instead, they are written as free-floating values (eligible to be GC'd) and commits made in detached-head mode are "dangling" commits: they are not associated with a branch (and also eligible to be GC'd). A typical pattern for using this mode is to set the database head, make some writes and commits, and land this work by writing into the dolt_branches table.
  • myDatabase_working: setting this variable only affects the working set of session, essentially force-setting it to some other state.
    This PR restricts the use of these variables in the following way:
  • myDatabase_head_ref: if the new value is a branch or working set ref, the session will switch working sets, other refs will error.
  • myDatabase_head: the new value will be resolved to a commit. If the commit corresponds to a single branch, the session will switch to the associated working set. If the commit cannot be resolved or doesn't correspond to a unique branch, an error will be returned. edit: this variable is now read-only
  • myDatabase_working: similar to _head, if the new value can be resolved to a unique working set, the session will switch to it, otherwise it will error. edit: this variable is now read-only
• 2935: merge constraints more intelligently
  Merging new checks defined on separate branches used to just drop both checks, and somehow NOT NULL constraints too. This fixes that.
  Uncertain behavior:
  • merging constraint and conflicting data is successful
  • modifying constraint and conflicting data is successful

go-mysql-server

• 881: More tests for alter table bugs
  1. Split PkOrdinals tests into DDL and DML (new)
  2. New script test that triggers the primary key / secondary key ordinal mismatch
  3. Add ctx.Done() checks to a few key node iterators before operating on child rows (may have missed some, but tried to hit the table, index, and edit iterators)
     companion PR: #3013
• 878: Fix logic with OR operation and add missing nil check to Like
  MySQL has some really weird behavior where the OR of two non bools evaluates too false. Improperly handling this situation was trickling up to a LIKE panic on the new query added to queries.go

  SELECT "A" OR "A";
  +------------+
  | "A" OR "A" |
  +------------+
  |          0 |
  +------------+
  

• 873: Add support for making fulltext return unsupported feature
  This pr addresses this issue
  Cc mysql docs
• 872: sql: expose convert native go value to value
  Expose convertToValue function from memory package as ConvertToValue.
  Return an error instead of panicing if the value type is not known.
• 871: Apply bindvars to subqueries
  Cherry pick from dolthub/go-mysql-server#795 to fix Subquery bindvars specficially.
• 869: Fixed invalid hex number type association from number to blob
  Fixes #2968
• 861: have methods for dolt to call to parse check expression
  To properly merge check constraints, Dolt needs to partially parse their expression strings.
  This PR gives Dolt access to some helper methods, to complete this operation. Should have no functional effect on GMS.
• 849: Implemented sort node, number type, and handler support for Row2, Expr2, Type2
• 838: Table function support
  Adds support for table_functions in go-mysql-server.
  Depends on:
  • dolthub/vitess#131

vitess

• 139: Revert "Merge pull request #137 from dolthub/james/special-comments"
  This reverts commit 19d17da876ea71be47f41092d8ab420885b9ba18, reversing
  changes made to 0bb5262822024a793f65040a77430ebe0624058e.
• 137: Handle subquery indexes correctly for special MySQL comments
  Fix for: #2980
  When Tokenizer detects a special MySQL comment, it just creates a new Tokenizer and embeds it in the old one under the specialComment member variable.
  This "inner" Tokenizer is then used to handle all the parsing, and its results are just passed to the outer Tokenizer. However, the issue is that the lexer or whatever only reads the Position member variable from the outermost Tokenizer.
  So my proposed fix is to just copy over the Position of the inner Tokenizer to the outer one (with an offset to handle the leading /*![12345]. I think a better fix might be to just replace the old Tokenizer with the one we create for specialComment?
• 136: Initial implementation of fulltext with indexes and DDL
  This pr
  1. Adds fulltext parsing support for indexes and DDL.
  2. Formats the repo
• 135: Allowing Event Keywords to be used as column name, table name, variable name, etc
• 134: Get Closer to MySQL's Behavior for Keywords (Reserved and Non-Reserved)
  I doubt I'll be able to get all of the keywords working immediately.
  This PR focuses on getting as many of the keywords in and working, and documenting what doesn't work yet.
• 133: reserve found and allow as column safe
• 131: Table function support
  Add support for table_functions as a new type of table factor in the dolthub/vitess SQL grammar.

Closed Issues

• 2854: Syntax error when using event as column alias
• 2964: Use a helper function to handle errors returned by functions in a defer statement?
• 2968: value out of range - insert 0x... into binary
• 2977: Composite Primary Key doesn't seem to work
• 2462: Support -a in dolt tag command
• 2960: Runtime error: invalid memory address or nil pointer dereference error when cloning a Dolt repository