Merged PRs

dolt

• 5677: dolt assist command
  This PR implements a new dolt assist command that connects to ChatGPT to assist in various dolt tasks, including running SQL queries. It's experimental and hidden (not included in docs) for now.
• 5675: go/doltcore/merge: improved schema merge tests
• 5672: go/libraries/doltcore/doltdb: durable/table: Always write the table value we are refing. Ensures it always makes it across the GC safepoint.
• 5670: Fix point lookup bug w/ prefix matching secondary
  re: #5648, test here: dolthub/go-mysql-server#1685
• 5667: go/libraries/doltcore/sqle/cluster: commithook.go: Fix initialization of commithook so it does not report isCaughtUp right after construction.
  Misreporting isCaughtUp would race to allow a primary to gracefully transition to standby without true-ing up the replicas immediately after creating a new database, for example.
• 5666: go/store/nbs: safeguard chunkJournal tablePersister methods for read-…
  …only journals
• 5664: go/doltcore/sqle: fix dropped error in read-replica-db
• 5660: support 'add' and 'remove' options for dolt_backup()
• 5649: updates sql-client to print query timing for more statement types
  fixes: #5518
• 5635: go/store/nbs: replace builtin golang map with swiss.Map
• 5601: go/store/nbs: make chunk journal lock NBS manifest while open
• 5385: go/store/nbs: Make the panic in the finalizer for an unclosed table file index always on.
  Add a flag to make it not take the stack trace at the time of index creation.
  Disable stack trace taking by default.
  Make all unit tests pass the sanity checking.

go-mysql-server

• 1702: sql/plan: track column renames on alter/modify column
  fixes #5689
• 1696: resolve recursive CTE in scalar subquery
  Re: #5631
• 1693: update separator parsing in group_concat
  Updates separator parsing for group_concat to use new Separator struct. This change is needed to allow '' as a separator.
  fixes: #5570
  related: dolthub/vitess#230
• 1692: don't push Filter below Limit
  When there's a query with a filter over a subquery with a limit, we incorrectly push filters down to the subquery.
  Example:
  This

  select * from (select * from t limit 1) t where i > 1;

  is not equivalent to

  select * from (select * from t where i > 1) t limit 1;

  Fix for: #5568
• 1690: Use ordinals to force stable TopN heap sort
  Re: #5640
  ... ORDER BY <expr> LIMIT 1 and ... ORDER BY <expr> LIMIT 1 OFFSET 1 both use TopN in a way that surprises users expecting stable output orders.
  For example, the second query returning the first row is within the SQL spec but not user friendly:

  -- setup
  create table xy (x int primary key, y int);
  insert into xy values (1,0),(2,0),(3,0),(4,0);
  -- expect first row
  select * from xy order by y asc limit 1;
  +---+------+
  | x | y    |
  +---+------+
  | 0 |    1 |
  +---+------+
  -- expect second row
  select * from xy order by y asc limit 1 offset 1;
  +---+------+
  | x | y    |
  +---+------+
  | 0 |    1 |
  +---+------+

  This PR adds ordinals to rows in the TopN heap to differentiate insertion order in sort-groupings.
• 1689: Make merge_join pushdown safe
  Index lookups were being pushed into merge join:

  tmp1> explain select /*+ merge_join(xyz,uv) join_order(xy,uv) */ * from xyz join uv on x = u where y = 1;
  +--------------------------------+
  | plan                           |
  +--------------------------------+
  | MergeJoin                      |
  |  ├─ cmp: (xyz.x = uv.u)        |
  |  ├─ IndexedTableAccess(xyz)    |
  |  │   ├─ index: [xyz.y]         |
  |  │   ├─ filters: [{[1, 1]}]    |
  |  │   └─ columns: [x y z]       |
  |  └─ IndexedTableAccess(uv)     |
  |      ├─ index: [uv.u]          |
  |      ├─ filters: [{[NULL, ∞)}] |
  |      └─ columns: [u v]         |
  +--------------------------------+

  In rare cases like above, this pushes an unsafe index into merge join that is not sorted monotonically on the join condition. A merge join index that violates join condition monotonicity will miss matches. In even rarer cases, the out of order value for the join condition included NULLs, causing this error: #5633.
• 1688: Support for recursive CTEs with no union
  Ex:

  with recursive a as (select 1) select * from a union select * from a;

  Re: #5657
• 1687: Fix left merge join bug
  Re: #5652
• 1686: Fix issue where special characters within JSON were encoded.
• 1684: Fix hoisted scope renaming error
  Disambiguating a hoisted scope requires renaming all references. We were missing intra-scope filter renames.
  re: #5654
• 1683: Fix distinctness/sort enforcing
  The original bug: #5651 duplicates a RIGHT_SEMI_LOOKUP_JOIN row because we were distincting right full row rather than the subset of join attributes.
  This PR adds some more tests around ordering and sort enforcing in the memo.
  The overview is that DISTINCT is weird because it is something in-between a property of a relational expression and the property of a relational group. It is an enforcer that we can implement as an ORDERED_DISTINCT or ditch altogether when child nodes provide supportive sort orders. We could imagine bifurcating the memo into buckets, with expression groups sectioned into groups based on sort orders, and costing considering the cardinality of children plus conditional sort enforcers. More work needed to think through how PG and CRDB do this generally.
• 1682: Rework how we create JSON strings to match MySQL.
  Fixes #4499
• 1680: Fix partial join hints
  ResolvedTable and TableAlias must carry comments if we want to hint partial joins created by unnesting subqueries.
• 1679: Small fix for a join-related regression in Dolt
• 1670: Adding a new GH action to label incoming issues and PRs from customers
  This same workflow has been running in the dolt repo for a couple weeks now. This PR rolls it out to go-mysql-server.
  The intent is to label issues and pull requests from customers to help us see them more easily in all the other issues and pull requests we create.
• 1519: partially implementing st_within
  Apparently, I've already added support for st_within() for Point vs Geometry, so this is just adds a bunch of test cases for every other geometry.
• 1512: partially implementing st_equals
  MySQL docs for reference: https://dev.mysql.com/doc/refman/8.0/en/spatial-relation-functions-object-shapes.html#function_st-equals
  This adds support for st_equals() only in the Point vs Point case. There are skipped tests for all other cases.

vitess

• 232: go/mysql/conn.go: Improve server-side cursor support.
  1. Send LastRowSent with the result set batch in which the cursor is actually exhausted.
  2. Only send CursorExists in the response to COM_STMT_EXECUTE which creates the cursor.
• 230: update separator parsing for group_concat
  Updates separator parsing for group_concat to indicate whether the default separator should be used. This change is needed to allow '' as a separator.
  related: dolthub/go-mysql-server#1693
• 229: Added support for ALTER TABLE ... COLLATE
  Fixes #5623
• 228: simulate server side cursors
  To be able to use the newest MySQL foreign data wrapper for Postgres, we have to support server side cursors.
  This PR emulates them, and hopefully gets us far enough...
  Additionally, this includes a fix for Windows clients (and other clients that don't have the DeprecatedEOF client capability flag set) where we send an extra EOF packet when we shouldn't.
  Fix for: #5441
  And this: #3029
  This one too (kinda): #4840
• 227: add create and drop event parsing
  Add support parsing CREATE EVENT and DROP EVENT statements.

Closed Issues

• 5689: Crash when trying column change
• 5676: Intellectual property rights and Patent Number for the Dolt DB Storage method.
• 5243: Implement add/remove backup in dolt_backup() stored procedure
• 5570: GROUP_CONCAT() parses separator but does not respect it
• 5643: Incorrect number of results returned from INNER JOIN
• 5639: Join dropping results
• 5657: dolt panics when recursive CTE is used
• 5651: RIGHT_SEMI_LOOKUP_JOIN duplicating join rows
• 5652: Left merge join drops a row
• 5648: Point lookups should match prefix
• 5633: MERGE_JOIN nil comparison found in operand
• 5632: Scope unnesting field index bug.
• 5654: "Field is not on schema" error during subquery unnesting
• 5638: IN_TUPLE filters subject to range scan duplication
• 5650: dolt diff/patch doesn't produce valid SQL for binary columns
• 4499: sql.JsonDocument ToString returns strings without whitespaces
• 5658: Recursive CTE column missing from schema
• 5518: Write queries don't report query time in the dolt sql-client

Latency