Backwards Incompatible Changes

We have been testing and matching MySQL's behavior for different types including enum. We were also far too permissive on which columns were allowed to be auto_increment. Dolt is now more strict with the types and constraints it allows you to define. It is also more strict with auto_increment.

Per Dolt’s versioning policy, this is a minor version bump because these changes may impact existing applications. Please reach out to us on GitHub or Discord if you have questions or need help with any of these changes.

Merged PRs

dolt

• 9505: Dolt diff --system flag
  Adds a --system flag to dolt diff that allows you to display system tables, such as the following: Dolt ci tables like dolt_ci_workflows and dolt_ci_workflow_jobs, and dolt_ignore.
• 9492: #9481 - Move auto_increment validation to GMS
  Remove database-side auto_increment type validation to allow GMS to handle all validation logic.
  Fixes #9481
  • Removed ErrNonAutoIncType error from col_coll.go
  • Removed auto_increment validation logic from ValidateForInsert in schema_impl.go
  • Removed unused helper functions isAutoIncrementKind and isAutoIncrementType
  • Added comprehensive tests for auto_increment on various data types
• 9488: Optimize dolt_diff_ tables for lookups on primary key ranges.
  Fortunately, 90% of the work for this happened to have been done last week with #9419
  The dolt_diff_ tables expose an index that matches the primary keys of the underlying table. We even already have tests that this index gets chosen for select expressions that limit the key to a specific value or range.
  However, in practice this index wasn't actually used.
  As of this PR, the index will actually be used to optimize the ranges that are scanned at each commit.
  I'm not sure of an obvious way to test this: ordinarily I'd add a test that a query selects the index and returns the correct answer (and I added such a test for the index on the from_ keys)... but the engine was already using the index and then discarding it.
• 9476: go/store/nbs: Improve GC behavior when non-persisted changes have been made to the store.
  GC has a fast-path check so that it performs a noop if there have been no changes to the database since the last time GC ran successfully. This change improves those checks so that GC works appropriately in more cases. In particular:
  1. When GC is running against a database that has in-flight writes, it will now still run, potentially collecting garbage created by the inflight writes, instead of not doing anything.
  2. When GC is running against a database that has a journal file, but where the journal file has never been persisted into the state of the manifest of the database because it only has inflight writes which have never committed or updated the root, GC will now run appropriately and will collect the contents of the journal file if it is appropriate to do so.
• 9436: Improve dolt checkout error handling for missing tables
  Fix #9392

  Summary

  Enhances the dolt_checkout stored procedure to handle missing table specifications
  more gracefully by processing valid tables while providing detailed warnings for
  invalid ones, instead of failing immediately on the first missing table.

  Changes Made

  • Enhanced Error Handling: Modified checkoutTablesFromHead() to collect warnings
    for missing tables instead of failing fast
  • Improved User Experience: Valid tables are now processed successfully even when
    some specified tables don't exist
  • Better Error Reporting: Multiple missing tables now show aggregated error
    messages
  • Added Test Coverage: Comprehensive tests for single and multiple missing table
    scenarios

  Behavior Changes

  Before: call dolt checkout valid_table missing_table would fail entirely
  After: call dolt_checkout valid_table missing_table processes valid_table
  and returns warning about missing_table
• 9435: go: statspro: Small cleanups in error handling and session command handling.

go-mysql-server

• 3093: go.mod: Bump to Go 1.24.0.
• 3092: add UTC() to unix time min and max Timestamp
• 3091: sql/types: JSONWrapper: Thread Context on some methods on JSON-related types, such as ToInterface and Compare.
• 3090: pull from main
• 3089: DECIMALS with foreign keys tests
  This PR adds skipped tests for decimals in foreign keys.
• 3087: Mixed Type Foreign Key
  This PR adds some tests for foreign keys where the parent and child columns are of different types, some of which are actually compatible in MySQL.
• 3085: Fix enum DEFAULT NULL validation in analyzer
  Fixes #9491
  Fix enum DEFAULT NULL validation
• 3084: #9481 - Move auto_increment validation to GMS
  Move auto_increment constraint validation entirely from Dolt to go-mysql-server for better separation of concerns and MySQL compatibility.
  Fixes #9481
  Fixes #9470
  • Add validateAutoIncrementType() function using existing type checking
  • Fix validation order to prioritize auto_increment before index validation
  • Unskip comprehensive test coverage for all invalid data types
  • Fix bugs: YEAR and BIT types were incorrectly allowed
• 3083: Split function-specific engine test queries into their own file
• 3082: Allow GroupBy primary key (and some test clean up)
  related to #4998
  moved some passing tests from #1765, moved some queries to BrokenQueries section
  added tests for #7095
• 3081: make_set and export_set functions
  Documented here.
  https://dev.mysql.com/doc/refman/8.4/en/string-functions.html
  Already gave this a once-over, but see what you think.
• 3080: add auto_increment tests with various types
  This PR adds a bunch of skipped tests demonstrating MySQL auto_increment behavior.
• 3079: INSERT string function
• 3078: #8893 - Fix charset validation to match MySQL behavior for issue #8893
  Fixes #8893
  • Fix charset validation error messages to show proper column names and row numbers.
  • Add MySQL-compatible strict/non-strict mode behavior for invalid UTF-8 handling
  • Preserve charset validation in ConvertToBytes() method (data insertion)
  • Remove charset validation from SQL() method (query results)
    This allows querying existing invalid UTF-8 data (enabling cleanup) while preventing new invalid data insertion with proper error messages.
• 3077: Add various SET type tests
  This PR adds a bunch of tests for SET and ENUM behavior, skipping ones that are currently broken.
  Additionally, refactors/deduplicates the the tests.
• 3076: #9425 - Fix enum zero validation in strict mode
  Fixes #9425:
  • Add strict mode check for 0 values in EnumType.Convert()
  • Return data truncation error for invalid 0 values in strict mode
  • Allow 0 values when empty string is explicitly defined as enum value
  • Add row number tracking in insertIter for accurate error reporting
  • Enhance enum errors with column name and row number
  • Fix ErrInvalidType formatting issues in enum expression
    🤖 Generated with Claude Code
• 3074: #9424 - Fix enum foreign key constraints to match MySQL behavior
  Fixes #9424
  • Allow enum types to reference each other in foreign keys regardless of string values
  • MySQL allows enum foreign keys to match based on underlying numeric values
  • Modified foreignKeyComparableTypes to handle enum types specially
  • Updated test expectations to use correct error types (ErrForeignKeyChildViolation)
    🤖 Generated with Claude Code
• 3070: Don't prune VirtualColumnTable tables
  fixes #8968
  Pruning columns in VirtualColumnTable tables causes indexing errors when VirtualColumnTable Projections are evaluated
  Future work: prune VirtualColumnTable Projections so that VirtualColumnTable underlying table can be pruned
• 3068: #9426 - Support enum string context in functions
  Fixes #9426
  • Modified ConvertToCollatedString in sql/types/strings.go to handle enum types
  • Updated CONCAT function to use type-aware string conversion
  • Enabled and fixed "enum conversion to strings" test
    🤖 Generated with Claude Code
• 3064: Abstract IsNull and IsNotNull expression logic
  Postgres has slightly different behavior from MySQL for IS NULL and IS NOT NULL expressions. In Postgres, records and composites need to have their individual fields checked as part of these expressions. Additionally, in Postgres, for records and composites, IS NOT NULL is not equivalent to NOT(IS NULL). For those types in Postgres, IS NULL returns true when all values in the record or composite are NULL and IS NOT NULL returns true when all values in the record or composite are not NULL. For example, both ROW(NULL, 1) IS NULL and ROW(NULL, 1) IS NOT NULL return false.
  To enable Doltgres to support this difference in behavior, this PR creates interfaces for IS NULL and IS NOT NULL expressions, as well as a new factory type that enables the analyzer to create new instances of those customized expression implementations.
  This PR also reverts changes from dolthub/go-mysql-server#3004.

Closed Issues

• 9473: Incorrect string conversion for ENUMs in CASE statements
• 8893: Invalid string for charset utf8mb4
• 9471: Numeric DEFAULT values are not allowed in SET
• 9392: dolt checkout test0 test1 fails all if one is missing, but git checks out available ones
• 9481: Prevent auto_increment constraint on every type other than INT
• 9470: SET column types cannot be auto_increment
• 9425: 0 value is not allowed for enum
• 6458: sql_mode=only_full_group_by bugs
• 9447: Automatic GC causes large database sizes for subsequent failed transactions

Performance