Merged PRs

dolt

• 7644: go/store/datas/pull: pull_chunk_fetcher: Move chunk fetching to a streaming interface instead of batch.
  We want to better pipeline I/O when pulling from a remote, and moving a streaming interface where storage can see more of the addresses that are needed at once will allow us to achieve it.
  For now, we implement the streaming interface just by calling the existing batch get interface.
• 7643: [no release notes] odd n ends clean up
  Random things I cleaned up while digging into NBS. All NoOp.
• 7637: innodb_autoinc_lock_mode tests
  companion pr: dolthub/go-mysql-server#2410
• 7629: refactor tablespec
  Use changes to handle table option parsing, and apply auto_increment table option.
  Additionally, changes resetScripts() in doltHarness to not drop and recreate all auto_increment tables.
  This step is not necessary, and causes problems now because we actually display and read AUTO_INCREMENT table option in show create table statements.
  Contains changes from: #7631
  Companion PR: dolthub/go-mysql-server#2401
• 7582: add dolt_hashof_table func and dolt_hashof (Deprecate hashof)
  I created this function for my GDC demo. We can talk about pros and cons of integrating this back to main.
• 7579: use system variable interface
  Depends on GMS PR: dolthub/go-mysql-server#2375
• 7572: go/store/datas/pull: pull_table_file_writer.go: Optimize pull/push to allow for concurrent table file uploads.
  Move management of the upload workers and the table file writer to its own struct which is testable in isolation.

go-mysql-server

• 2412: New interface for binlog primary callbacks
  First pass on connecting the GMS layer with the Dolt layer for handling callbacks when the SQL server is acting in binlog primary mode, through the new BinlogPrimaryController interface. This new interface pretty closely mirrors the existing callback interface for replica callbacks, the BinlogReplicaController interface.
  Related to #7512
• 2411: implement json_search()
  MySQL Docs: https://dev.mysql.com/doc/refman/8.3/en/json-search-functions.html#function_json-search
• 2410: Adding system variable innodb_autoinc_lock_mode
  We currently only support innodb_autoinc_lock_mode = 2, not 0 or 1.
  MySQL Docs:
  https://dev.mysql.com/doc/refman/8.0/en/innodb-auto-increment-handling.html
  related: #7634
• 2404: Improve handling of unsigned and decimal types in JSON
  Fixes dolthub/go-mysql-server#2391
  MySQL's JSON type differs from standard JSON in some important ways. It supports types not supported in standard JSON, such as separate types for integers and floats, an unsigned int type, and a decimal type.
  Prior to this PR, we would convert values to JSON by using the encodings/json package to marshall the value to a JSON string and then unmarshall it to a go map. This is not only slow, but it's incorrect for these additional types.
  The main purpose of this PR is to add special handling for these types that allow them to be stored in JSON documents. We also avoid generating and parsing JSON in places where it's not actually necessary, and fix bugs where decimals get incorrectly converted into strings, or unsigned ints get converted into signed ints.
  Finally, this fixes an issue where we send incorrect bytes for JSON-wrapped decimal values along the wire.
• 2403: fix dbName not being used in the example
  This PR makes sure that dbName in the example is actually being used, instead of having a hardcoded "mydb" in createTestDatabase.
  fixes #2402
• 2401: refactor and parse table options, support auto_increment table option
  Table Options are now parsed as structs, so we can read/use some of the variables.
  Character Sets, Collations, Create Table, TableSpec, etc. have been refactored.
  Additionally, this PR adds support to parse and use the auto_increment table option.
  TODO:
  • CREATE TABLE ... LIKE ... needs to preserve table opts, like comments
  • alter table add column ... auto_increment does not work when there are already rows
    Companion PR: dolthub/vitess#322
• 2399: fix custom insert ordering for pk
  fixes dolthub/go-mysql-server#2397
• 2398: fix in-memory implementation of RenameTable to read from session
  The in-memory implementation of RenameTable uses data from the BaseDatabase, instead of reading it from the session.
  This is problematic when there are multiple alter statements.
  Additonally, includes some small refactor so all functions are pointer receiver instead of a mix.
  fixes dolthub/go-mysql-server#2396
• 2394: Bug fix: Set non-boolean system variable enum values to 'ON' or 'OFF'
  We were automatically converting ON and OFF values to to true and false when setting a system variable, which made it impossible to set system variables to those enum values. For example:

  SET @@GLOBAL.gtid_mode='ON';
  Variable 'gtid_mode' can't be set to the value of 'true'

• 2393: Restored and refactored missing functionality required by Dolt
• 2375: create system variable interface
  This PR creates SystemVariable and SystemVariableScope interfaces.
  This allows doltgres to use these interfaces for defining and handling all the configuration parameters.
  The SystemVariable struct is renamed to MysqlSystemVariable.
  The SystemVariableScope byte is renamed to MysqlSVScopeType.

vitess

• 324: Ensure that float values parsed into expressions will always be parsed back into floats, not decimals.
• 323: Parser support for SOURCE_AUTO_POSITION
  When acting as a replica, Dolt implicitly assumes SOURCE_AUTO_POSITION is set to 1. This adds parser support so that it can be explicitly specified. Adding this so that when we're configuring a MySQL replica, we can use the same setup code without having to special case this parameter.
• 322: parse table options into struct
  Instead of just making a large string that is reparsed in GMS, parse table_options into a list of structs similar to how we handle index_options and constraint_options.
• 321: [ast] walk tableFuncExpr for bind variables
• 320: Updates for binlog primary protocol
  • Porting over more code from vitess.io/vitess – support for the COM_REGISTER_REPLICA command.
  • Fixing a bug in Vitess' deserialization of the COM_BINLOG_DUMP_GTID command.
  • Adding some String functions to help with debugging.
  • Exposing the ability to flush a connection's buffer, as a short-term workaround for needing to flush the buffer more frequently than at the end of the connection in order to support `COM_BINLOG_DUMP_GTID'.
• 317: Port: Support for Vitess server to send binlog events
  Porting over support from the main Vitess repo for a server to send back binlog events over a connection. Also includes support for handling the COM_BINLOG_DUMP_GTID command.
  Related to #7512

Closed Issues

• 7641: Does it support connecting to existing MySQL or linking to TDSQL
• 5486: Ability to "project" schema at a commit onto data at another commit
• 7630: Add topic 'mariadb' to GitHub repository
• 7621: Migrate dolt gc to SQL
• 2402: Consistent usage of variables in the example
• 2391: Potential regression: number cast to JSON no longer read as float
• 2396: Running multiple migrations in a transaction
• 2397: Primary key column order changes column order on insert

Performance