Merged PRs

dolt

• 10919: Sort adaptive encoding columns by size for out-of-band storage
  Note: This will currently change how we write rows with adaptive encoding columns. Rows written on older versions of Dolt can still be read, but writing them back will result in a new chunk hash. But since Dolt currently doesn't use adaptive encoding and Doltgres is in beta, that's acceptable.
  Claude's Summary
  • BuildPermissive now sorts adaptive encoding columns by size savings (largest first) before deciding which to move out-of-band, instead of visiting columns in sequential order
  • This keeps smaller values inline when possible, producing shorter tuples when columns have varying sizes
  • Added two new test cases verifying the largest-first ordering with 2 and 3 columns of different sizes
    Test plan
  • Existing TestTupleBuilderAdaptiveEncodings tests pass (including inline one of two columns)
  • New test largest value moved out-of-band first: two columns where column 0 is small and column 1 is large — verifies column 1 goes out-of-band while column 0 stays inline
  • New test three columns largest first ordering: three columns (medium, small, large) — verifies only the largest goes out-of-band
  • Full store/val package tests pass
• 10915: Fix fsck use earl.Parse to correctly resolve database path cross-platform
  fsck.go was using url.Parse to parse the database file URL. On Windows, url.Parse places the drive letter into the URL Host field rather than Path, causing dbfactory/file.go to construct an invalid file path.
• 10910: Threaded context through all GMS interfaces
  Threads the context as added by:
  • dolthub/go-mysql-server#3513
• 10908: go: remotestorage: reliable/http: Fix some misbehavior when ChunkFetcher is working against HTTP/2 endpoints.
  HTTP/2 frames responses differently from HTTP/1 and we see io.EOF on the response at slightly different times. A race in ChunkFetcher meant that it treated read-to-end-of-range-caller-wanted-and-was-Closed as a failure if the Close came in before the io.EOF from the read against the http.Response.Body.
  Fix that race and fix some behavior around our fetch health tracking related to it.
• 10906: Fix UPDATE when table has a BINARY primary key and a foreign key to use bytes.Equal
  Updating a row in a table with a BINARY or VARBINARY primarkey key and a foreign key caused a runtime panic. Any secondary index on such a table (including the index backing a foreign key constraint) would trigger it.
  • Fix isNoopUpdate to use bytes.Equal when comparing BINARY/VARBINARY column values
    Fix #10903
• 10893: Fix auto increment state after backup restore
  Fix database restored with dolt backup restore could generate incorrect AUTO_INCREMENT values when accessed through a still-running dolt sql-server.
  The restored table persisted the correct auto-increment value, but the server’s in-memory auto-increment tracker could be missing state for the restored database. The tracker now initializes missing table state from the restored table before allocating ids.
  Fix #10892
• 10889: use table and schema when adding/removing foreign key for Doltgres
  Depends on: dolthub/go-mysql-server#3514
• 10884: go: utils/config: file_config.go: Make concurrent reads and writes safe.
  Persisted SQL session variables land in at an instance of *FileConfig. It needs to be thread-safe.
  Fixes some transient crashes which can occur while under load and switching dolt_cluster role, for example.
• 10874: Feature: Indexed functional expressions
  Adds support for secondary indexes to contain functional expressions, which are then used to optimize queries with filters and join conditions that use that indexed functional expression.
  Initial performance testing between Dolt and MySQL shows that Dolt matches, or in some cases beats, MySQL's performance when these indexes are used.
  This initial implementation only supports a single expression in a functional index. Next steps for this work after this PR are:
  1. adding support to Doltgres
  2. extending support to multiple expressions in an index (functional expressions mixed with column names)
  3. add a sysbench test to compare query performance with indexed expression for Dolt and MySQL
• 10831: Preserve user GIT_SSH_COMMAND and block interactive SSH prompts without clobbering env
  When dolt pushes or fetches over a git+ssh remote, it was unconditionally overwriting the user's GIT_SSH_COMMAND environment variable with its own value (ssh -o BatchMode=yes). This broke workflows where users set GIT_SSH_COMMAND to pick a specific key.
  • GIT_SSH_COMMAND set by the user before invoking dolt is now preserved; dolt no longer overwrites it
  • Git subprocesses are now started in a new session (setsid on Linux/macOS, DETACHED_PROCESS on Windows) so SSH cannot reach the terminal to prompt for a passphrase or host key confirmation
  • Regression tests added against a real OpenSSH daemon
    Fix #10811

go-mysql-server

• 3516: Fix SHOW CREATE VIEW fail when underlying table is dropped
  MySQL returns a view's stored definition regardless of whether its underlying table exists. GMS re-resolved the view body at query time, which failed if the referenced table had been dropped.
  • Split SHOW CREATE TABLE and SHOW CREATE VIEW into separate build paths.
  • SHOW CREATE VIEW now reads the stored definition directly via ViewDatabase.GetViewDefinition
  • Extract showTargetInfo helper to deduplicate db/table/asOf extraction
    Fix #10902
    Blocked by dolthub/vitess#461
• 3514: handle foreign key errors for Doltgres
• 3513: Threaded context everywhere
  Integrators need to access the sql.Context in locations that it was not previously passed to, which has resulted in workarounds in some cases (such as storing the context in nodes) or missing functionality altogether (if a context is needed during function creation for example). This PR makes it such that the context is effectively passed everywhere.
  The one singular exception is in the String() function on nodes and expressions. Since those take advantage of the fmt.Stringer interface, they cannot accept a context without breaking that contract (which would also break %s and %v in the fmt package). This isn't considered too large an issue as strings are generally used for debugging, however if it is a core feature used by an integrator, then they'll still have to resort to workarounds.
• 3511: cache arithmetic type
  We spend a large amount of time determining the correct resulting expression type for Arithmetic expressions.
  Once the resulting type is determined for an math operation, it doesn't change for the remainder of the rows, so we should be able to cache this result and avoid many type checks and interface conversions.
• 3510: Ignore unique key errors when inserting into fulltext position table
  fixes #10882
  Ignore unique key errors when inserting into fulltext index.Position table (similar to how unique key errors are ignored for index.DocCount).
  We should consider using a non-unique index for index.Position and index.DocCount or restructuring how we're writing to these tables if we're always going to be ignoring unique key errors.
• 3491: Feature: Indexed functional expressions
  Adds support for secondary indexes to contain functional expressions, which are then used to optimize queries with filters and join conditions that use that indexed functional expression.
  Initial performance testing between Dolt and MySQL shows that Dolt matches, or in some cases beats, MySQL's performance when these indexes are used.
  This initial implementation only supports a single expression in a functional index. Next steps for this work after this PR are:
  1. adding support to Doltgres
  2. extending support to multiple expressions in an index (functional expressions mixed with column names)
  3. add a sysbench test to compare query performance with indexed expression for Dolt and MySQL

vitess

• 463: Relax error handling for COM_FIELD_LIST to avoid mysql client disconnecting
  The mysql client sends COM_FIELD_LIST commands by default (unless the -A option is given) to collect field metadata for all tables/views. If a view has a broken definition, then it was returning an error, which caused the mysql client to disconnect and reconnect. This change ignores errors when querying tables/views for a COM_FIELD_LIST command, which fixes the client disconnect issue and lets us match the MySQL server's behavior.
  GMS CI Tests: dolthub/go-mysql-server#3518
  Dolt CI Tests: #10912
  Fixes #10634
• 462: Added context to Injectable
  Needed for:
  • dolthub/go-mysql-server#3513
• 460: Add digit-leading and hex-prefixed column names in qualified dotted access
  Fix parsing of unquoted dotted column references where the column name starts with a digit or hex prefix. Both two-part and three part (db.table.col) forms are now supported.
  • Fix the tokenizer treating a dot followed by a digit as the start of a decimal literal despite the preceding token was an identifier.
  • Add numeric_col_id as a grammar non-terminal covering INTEGRAL and HEXNUM tokens, and wired it into the column_name rule.
  • Move skipped cases from TestNotWOrkingIdentifiersStartingWithNumbers into TestParsingIdentifiersStartingWithNumbers and extended coverage.
    Fix #10757
    Related dolthub/go-mysql-server#3494
• 459: fix trigger parsing
  Should use old_lexer_position instead of lexer_position to avoid token getting swallowed.

Closed Issues

• 10892: AUTO_INCREMENT value lost after dolt backup restore
• 10634: MySQL client is unusable when there's an invalid view in dolt db
• 10903: UPDATE panics with a binary primary key and some foreign key since 1.86.0
• 10811: Dolt clobbers GIT_SSH_COMMAND if it already exists in the environment
• 10902: SHOW CREATE VIEW fails if underlying structure changed
• 10728: dolt pull from dolthub spends a long time doing nothing