Merged PRs

dolt

• 10022: go: libraries/doltcore/dbfactory: Do not use EventsUnaryClientInterceptor when interacting with remotestorage.
  This events source is too detailed and emits too many events to be appropriate for our usage metrics. It is useful in an operational context for DoltHub and Hosted, but it should be used by default in Dolt itself. We will revisit this events source if we need it for internal infrastructure, but for now we should disable it.
• 10020: refactor: make dolt_diff_summary respect dolt_ignore patterns
  User Contributed PR: #9946
  Closes: #5861
• 10016: Fixed branch control insertion bug
  A user reported that their dolt_branch_control table was in a corrupted state, as they could not delete a row even though running the appropriate DELETE FROM dolt_branch_control would report that 1 row was affected. Running SELECT showed that it was still there.
  I was able to replicate a way of getting the table into that state, which is to insert the exact same row as an existing row but with a larger permission set. dolt_branch_control handles primary key conflicts differently than other tables, as we throw a conflict when a subset is inserted. For example, '%' will match all entries that 's%' can match, so 's%' is a subset of '%'. Of course, when taking permissions into account, 's%' may not be a subset if it has a broader permission set than '%', in which case we want to allow the insertion of 's%'.
  The failure case was with exact matches. An exact match is still a subset when the permissions are as restrictive or more restrictive than the original, however it's not a subset when the permissions are broader, and this would result in allowing the insertion of the exact match. We convert all expressions into a specialized integer format, and store those integers in a prefix tree. Since the format will be the exact same, we end up overwriting the old entry, which is an implicit deletion without the accompanying bookkeeping in the rest of the data structure. This is what causes the mismatch, and creates the failure state.
  To prevent this, we never allow insertion when there's an exact match, so you must use an UPDATE in such cases. This will prevent that failure mode from occurring.
• 9994: use unsafe pointer for TextStorage
  unsafe.Pointer gets strings from []byte without allocating memory.
  This is especially beneficial for operations that pass around strings.
  Dolt already uses unsafe.Pointers in other places and this is within an immutable context.
• 9989: Allow foreign keys on nonlocal tables
  This change allows users to add foreign key relations on nonlocal tables (table names that match entries in dolt_nonlocal_tables and resolve to tables on other branches).
  The biggest obstacle was that the foreign key verification logic operates on the DB directly, but resolving the references in the dolt_nonlocal_tables table requires a Dolt SQL engine. Because we now use the engine for most operations, the engine is guaranteed to exist, but it wasn't obvious how to allow the storage code to access the engine in a way that didn't break encapsulation or create dependency cycles.
  The way this PR accomplishes this is by creating a new interface called doltdb.TableResolver, which has the method GetDoltTableInsensitiveWithRoot, which can resolve table names at a supplied root value. This object can be instantiated by the Dolt Session and passed into the DB layer.
  I'm not thrilled about adding the extra confusingly similar methods to doltdb.Database, but hopefully the differences between them are clear.
• 9971: Relax column tag uniqueness requirement
  Instead of requiring column tags to be unique across all tables in a branch, this change only requires column tags to be unique within each table.
  Column tags are intended to be an internal only, implementation detail of Dolt, but if a conflict occurred (two columns in the same branch having the same column tag), then it blocked customers and often required expert help to resolve. This change greatly reduces the chance of hitting a column tag conflict, since it reduces the scope across which a column tag must be unique.
• 9946: #5861 refactor: make dolt_diff_summary respect dolt_ignore patterns

  Summary

  Makes dolt_diff_summary table function respect dolt_ignore patterns to match dolt diff command behavior.

  Changes

  • Added ignore pattern filtering to dolt_diff_summary
  • Tables matching dolt_ignore patterns are now filtered out
  • Added system table filtering for dolt_* prefixed tables
  • Applied filtering to both general and specific table queries
  • ensure dolt_ignore table itself is still shown in dolt_diff_summary

  Testing

  • Added 5 integration tests and 4 bats tests

  Questions/Misc

  @macneale4 I think I got this change good for #5861, finally (sorry took me awhile, went out of country and was busy with work to jump back on it immediately). two things to call out for clarity:
  • dolt_ignore table itself should be included dolt_diff_summary (like .gitignore, for example), right?
  • there are two failing bats tests, I'm not sure why exactly since couldn't repro on my side locally.
    Closes: #5861
• 9906: Implement sql.ValueRow
  This PR implements sql.ValueRow, which is a server output friendly version of sql.Row.
• 9477: Faster storage chunk iteration
  Currently we have two access patterns where we need to iterate over all chunks in a storage file: Archive/Unarchive and FSCK. In both cases, we are loading one chunk at a time and disk IO becomes a bottleneck. This change updates the iteration methods to iterate over the data blocks of table files and archive files such that we can load data in larger batches.
  There are not new tests for this. archive and fsck tests should cover this.
  Testing on a 500Mb database: 8.7s to fsck before change, 5.0s after change.

go-mysql-server

• 3284: Use IndexedTableAccess for Sorts over a SubqueryAlias
• 3283: #9316: Fix PK setting and add new fields for CREATE TABLE ... SELECT
  Fixes #9316
• 3282: /.github/workflows/bump-dependency.yaml: sanatize stuff
• 3280: Allow string truncation when casting to date
• 3278: Respect precision when casting to datetime
  Also explicitly cast datetime to max precision during comparisons
  • fixes broken ld tests (dolthub/ld#21555)
• 3248: Implement sql.ValueRow
sql.ValueRow is the reincarnation of sql.Row2.
  This is an optimization to the sqlengine that eliminates interface boxing by never placing variables into a []interface.
  sql.ValueRowIter is implemented by:
  • TransactionCommittingIter
  • TrackedRowIter
  • TableRowIter
  • FilterIter
    Comparison should only use CompareValue when left and right are both NumericType, so Integers, Floats, Decimal, Bit64, and Year types.

vitess

• 441: #9316: Add CREATE TABLE ... AS SELECT support
  Fixes #9316
  Companion dolthub/go-mysql-server#3283
• 438: /go.mod: add patch version
• 437: docker-entrypoint.sh: Add VERSIONING to non-reserved

Closed Issues

• 5861: dolt_diff_summary system table should hide tables ignored by dolt_ignore
• 9316: Support CREATE TABLE (...) AS SELECT
• 3273: how to disable some features?