github dolthub/dolt v1.87.0
1.87.0

4 hours ago

Backwards Incompatible Changes

Different author and committer additions introduced are not interpretable by old Dolt clients. If you attempt to set an explicit committer that is different from the author, this will create a new field in storage. The dolt_log system table and procedure schemas have also been changed to be static. Compatibility for older clients is not guaranteed for all flags, e.g. --show-signature. Furthermore, the new Dolt client requires the Dolt server to be updated to this latest version to access all expected columns.

Merged PRs

dolt

  • 10996: cleaned up binlog serialization logic
  • 10995: Fixed deserialization for adaptive types during binlog replication
  • 10993: Fix dolt rm --cached to allow unstaged working changes
    dolt rm --cached incorrectly rejected tables with unstaged working changes or tables already dropped from the working set.
    • --cached branch filters tables absent from working via HasTable before calling RemoveTables.
      Fix #10987
  • 10983: Fix dolt_log table and procedure schema to be static and --decoration behavior
    dolt_log now exposes a fixed 12-column schema across the system table and procedure. Parents and signature columns remain opt-in via projection for the system table, and arguments in the dolt_log() procedure.
    • --parents and --show-signature no longer mutate the result of schema
    • Make parents and signature columns populate only when projected for dolt_log system table
    • --decorate=auto is deprecated in the sql-server context since TTY cannot be determined
    • --decorate=auto in CLI now correctly detects interactive terminal
    • Update bats coverage to be bidirectional for the server and client testing in compatibility
      Blocked by dolthub/vitess#467
  • 10979: adaptive encoding tests for mysql client integration
  • 10975: Use TARGET_ROW_SIZE table attribute to guide how aggressively we move adaptive encoded values out-of-band.
    TARGET_ROW_SIZE is a table attribute that can be set on tables. Dolt interprets this value as a size in bytes. When building a tuple, we attempt to keep the tuple under this size by moving adaptive-encoded values out-of-band until the size of the tuple falls below the threshold.
    Because Dolt does not allow tuples larger than 64K, we can always store the target value in 16-bits.
    The GMS interfaces use a uint64 for TARGET_ROW_SIZE, since other storage backends wouldn't necessarily have the same limit. When setting a value for TARGET_ROW_SIZE, Dolt verifies that it fits in 16 bits.
    Currently, this value can only be set while creating the table and can't be modified afterward.
  • 10974: Db/git remote UI
    This PR updates git remotes to push a branch visible on the Git remote's UI that contains a file with some info about the git ref used as the dolt remote. Addresses #10525
  • 10969: go-sql-server-driver tests for large adaptive and out-of-band values
    These tests exercise the replication, pull, and GC behavior for various kinds of schemas with large values that may be stored out of band. These tests failed before the inclusion of address serialization bug fixes from #10966
    Depends on dolthub/go-mysql-server#3533
  • 10966: bug fix: write addresses field in tuples if they contain only adaptive encoded fields
  • 10964: .github: ci-lambdabats-unix-adaptive.yaml: Add lambdabats DOLT_USE_ADPATIVE_ENCODING=true runs.
  • 10962: /.github/workflows: fix maven link
  • 10959: When merging, use a tuple builder to ensure that we generate the correct representation for the merged row.
    This matters for adaptive encoding columns, whose correct representation depends on other values in the row.
  • 10958: Fix obscure race: make sure that RunF finishes before StopF in sqlServer.Start()
    This is a candidate for race conditions leading to nil pointer exceptions in some doltgres harness setups.
  • 10952: adaptive encoding for JSON columns
    This will be turned on for the 2.0 release.
  • 10941: build(deps): bump go.opentelemetry.io/otel from 1.40.0 to 1.41.0 in /go
    Bumps go.opentelemetry.io/otel from 1.40.0 to 1.41.0.
  • 10933: build(deps): bump fast-xml-parser from 5.5.9 to 5.7.2 in /.github/actions/ses-email-action
    Bumps fast-xml-parser from 5.5.9 to 5.7.2.
  • 10832: build(deps): bump go.opentelemetry.io/otel/sdk from 1.40.0 to 1.43.0 in /go
    Bumps go.opentelemetry.io/otel/sdk from 1.40.0 to 1.43.0.
  • 10827: build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.78.0 to 1.97.3 in /go
    Bumps github.com/aws/aws-sdk-go-v2/service/s3 from 1.78.0 to 1.97.3.
  • 10804: #10813 fix(checkout): enforce --no-overwrite-ignore in DOLT_CHECKOUT -b via direct SQL

    Summary

    When calling DOLT_CHECKOUT('-b', ...) directly via SQL (without --move), the --no-overwrite-ignore flag was silently ignored. CheckOverwrittenIgnoredTables was never called for the isMove=false path in checkoutNewBranch, so creating a branch from a non-HEAD start point would succeed even when ignored tables in the working set differed from the start point.
    The fix captures the current working roots before commitTransaction (which resets session state, making GetRoots return false afterward), then runs the ignored table check before switching the working set to the new branch.

    Changes

    • Call CheckOverwrittenIgnoredTables in the isMove=false path of checkoutNewBranch
    • Capture roots before commitTransaction to avoid session state reset
    • Add engine script tests and bats tests covering --no-overwrite-ignore and
      --overwrite-ignore with a non-HEAD start point
      Closes: #10813
  • 10801: build(deps): bump lodash from 4.17.23 to 4.18.1 in /integration-tests/mysql-client-tests/node
    Bumps lodash from 4.17.23 to 4.18.1.* 10104: Add separate author and committer support
    Dolt now supports separate author (original intent) and committer (actual writer) identity for Dolt commits! Expect commit, cherry-pick, revert and merge operations (including SQL procedures) to pick up different author and committers automatically. The dolt_log, dolt_commits, dolt_diff, dolt_column_diff, dolt_branches, dolt_remote_branches system tables and respective table functions have been updated with author columns too.

    ⚠️ The committer columns in system tables or functions now show the committer identity instead of author. In old Dolt versions this was the result of no committer identity existing in storage, but has since been updated. We maintain backward-compatibility with this old format by duplicating the author identity into the committer identity. At serialization, however, the committer is only stored if the name or email is different from the author identity, allowing us to maintain the same commit hash history.
    ⚠️ We've appended the author columns under to end of each table to prevent issues with older Dolt clients.

New environment variables for committer identity are now available: DOLT_COMMITTER_NAME, DOLT_COMMITTER_EMAIL, DOLT_COMMITTER_DATE which take inspiration from their git counterparts.

⚠️ Environment variables (not SQL system variables) are only interpretable by the Dolt client. MySQL or alternative SQL clients will not pick these Dolt-specific environment variables up automatically in your terminal.

export DOLT_COMMITTER_NAME="Committer User"
export DOLT_COMMITTER_EMAIL="committer@example.com"
export DOLT_COMMITTER_DATE="2025-01-01T12:00:00Z"
dolt commit -m "Commit with custom committer date"
+----------------------------------+----------------+-----------------------+---------------------+--------------------------------------+--------------+-----------+------------------+---------------------+
| commit_hash                      | committer      | email                 | date                | message                              | commit_order | author    | author_email     | author_date         |
+----------------------------------+----------------+-----------------------+---------------------+--------------------------------------+--------------+-----------+------------------+---------------------+
| tdf81ch4ug4c9uc2p78rls1iihj95unn | Committer User | committer@example.com | 2025-01-01 12:00:00 | Commit with custom committer date    | 4            | Test User | test@example.com | 2025-11-19 10:15:47 |
| 0gonkh11sf1esld53b66jdpbu78sdq6i | Committer User | committer@example.com | 2025-11-19 10:15:43 | Add Charlie with different committer | 3            | Test User | test@example.com | 2025-11-19 10:15:43 |
+----------------------------------+----------------+-----------------------+---------------------+--------------------------------------+--------------+-----------+------------------+---------------------+

Since non-Dolt clients cannot interpret these environment variables, we've also added the system variables dolt_committer_name, dolt_committer_email, dolt_committer_date, dolt_committer_author, dolt_committer_email, dolt_committer_date. They enable committer and author identity to travel over the wire.

  • New environment variables for committer identity: DOLT_COMMITTER_NAME, DOLT_COMMITTER_EMAIL, and DOLT_COMMITTER_DATE
  • New session variables for author and committer identity: dolt_committer_name, dolt_committer_email, dolt_committer_date, dolt_committer_author, dolt_committer_email, dolt_committer_date
  • Commit related operations now honor the committer identity system variables over the wire: merge, cherry-pick, commit, revert, DOLT_TAG() etc.
  • dolt clients seed session identity variables from the dolt config on loopback connections, or the SQL client identity for non-loopback
  • Append author columns to dolt_diff, dolt_column_diff, dolt_branches, dolt_remote_branches, dolt_history_* system table, views and table functions
  • Signed commits use a V2 payload that appends CommitterName, CommitterEmail, and CommitterDate after the author fields, using the prior payload as a prefix for forward compatibility
  • Commit flatbuffer gained committer_name and committer_email fields, written only when they differ from author
  • CommitDate struct introduced to explicitly resolve author and committer timestamps at serialization unless otherwise specified by version control operations
    Fix #1374
    Blocks dolthub/doltgresql#2039
    Blocks dolthub/docs#2826

go-mysql-server

  • 3535: Enforce inline column references as foreign key constraints
    Fix #10970
  • 3533: fixed context reassignment error, which causes a panic in a running d…
    …olt SQL server
  • 3528: Bump go.opentelemetry.io/otel from 1.39.0 to 1.41.0
    Bumps go.opentelemetry.io/otel from 1.39.0 to 1.41.0.
  • 3527: Truncate invalid UTF-8 on INSERT IGNORE and warn on LIKE with bad charset pattern
    • INSERT IGNORE with invalid UTF-8 in a utf8mb4 column now truncates at the first bad byte
    • LIKE with an invalid UTF-8 pattern emits warning 1300 and return no match
    • Exported TruncateInvalidUTF8 from sql/types
      Fix #10924
      Blocks #10926
  • 3523: Add handling of TargetRowSize table attribute to GMS interfaces
    TARGET_ROW_SIZE is a custom table attribute whose exact meaning is implementation-defined by whatever storage backend implements it. It's used by Dolt to describe the threshold at which adaptive-encoded columns should be moved to out-of-band storage instead of being stored inline in the table.

vitess

  • 467: Add ERWarnDeprecatedSyntax 1287
    Blocks #10983
  • 466: Add parser support for multiple functional expressions in an index
  • 464: Add TARGET_ROW_SIZE table attribute to parser
    In order to allow users to control how aggressively adaptive encoded values get outlined, we're adding an additional table attribute.
    Our adaptive encoding design is based on Postgres's TOAST types. Postgres has two table attributes, TOAST_TUPLE_THRESHOLD, and TOAST_TUPLE_TARGET. If a row in Postgres is greater than TOAST_TUPLE_THRESHOLD, then Postgres will attempt to move values out-of-band until the row length falls below TOAST_TUPLE_TARGET.
    There doesn't seem to be an obvious benefit for having two separate values here, so I think we can just add one. We could reuse either name to copy Postgres, but it would be potentially confusing to call our table attribute TOAST_TUPLE_TARGET since we don't call the feature TOAST anywhere else.
    This PR names the new table attribute TARGET_ROW_SIZE, but also accepts TOAST_TUPLE_TARGET as an alias in order to match Postgres.
    Since we want to avoid adding business logic to the parser, the parser does not do the aliasing, it just parses both names and allows GMS to do the aliasing. I'm also open to having the parser responsible for aliasing if we have strong opinions.

Closed Issues

  • 10987: dolt rm --cached rejects unstaged changes; git rm --cached does not
  • 10986: docs: add note that Bitbucket does not support storing dolt metadata in "refs/dolt/data" for git-backed dolt remotes
  • 10970: Default FK enforcement differs for PRIMARY KEY vs UNIQUE parent refs
  • 1374: Add "Commiter" and "Commit Date" to Commit Meta
  • 10961: dolt_rebase drops replayed index on successful rebase of disjoint index additions
  • 2945: Add support for function indexes

Don't miss a new dolt release

NewReleases is sending notifications on new releases.