Merged PRs

dolt

• 2961: Fix error muting by defer and excessive memory allocation in newBSChunkSource. Fix localbs scheme using file scheme.
  Root issue is that blobstore.GetBytes returns a byte buffer whose capacity is too large. That buffer is passed to parseTableIndex which takes ownership of that buffer.
  In Dolt v0.37.0, parseTableIndex was changed to throw an error if the capacity or length of the given buffer is too large. This was done to prevent large memory usage by Dolt. Since the above buffer was too large, parseTableIndex would throw an error and the chunkSource would fail to initialize. Two other issues prevented this error from being caught.
  This bug was not caught prior to Dolt v0.37.0's release because errors returned in newBSChunkSource were being overridden by a defer. This muted any errors returned by newBSChunkSource including the error that parseTableIndex was throwing.
  Additionally, the bats tests for the localbs scheme were using the file scheme under the hood. This prevented the tests from catching this bug.
  This PR fixes the above issues and closes #2960
• 2958: Escape the given tablename in the engine_table_reader
  Fixes #2949
• 2956: store/datas: CommitOptions: Remove types references.
• 2954: performance/utils/sysbench_runner: fix dropped errors
  This fixes three dropped err variables in performance/utils/sysbench_runner.
• 2953: Bump GoMod and Trim the sql-use.expect
  ☕ An Automated Dependency Version Bump PR 👑

  Initial Changes

  The initial changes contained in this PR were produced by go geting the dependency.

  $ cd ./go
  $ go get github.com/dolthub/<dependency>/go@<commit>
  

  Before Merging

  This PR must have passing CI and a review before merging.
• 2950: Skipped bats test for updating tables with dashes in the name using dolt table import -u
  Issue #2949
• 2946: go/libraries/doltcore/dbfactory: aws.go: Support / as a character in database names for AWS remotes.
  This was previously forbidden because of how table files were stored in the
  destination bucket, but that was changed long ago and this is now well
  supported. Our internal remotes serving infrastructure uses chunk stores
  constructed in the same way.
• 2942: Fix Pk ordinal delete bug
  Consider a table with a primary key defined in a different order than the schema:

  > create table a (x int, y int, primary key (y,x);
  > insert into a values (0,1), (2,3);
  Query OK, 1 row affected

  Deletes were matching rows in the index and correctly tracking the affected row count, but generating intermediate schema-order key tuples incompatible with the table PK order. The incompatible keys failed to match the source keys in the roundtrip as the edits were applied to the table:

  > delete from a where x = 0;
  Query OK, 1 row affected
  > delete from a where x = 0;
  Query OK, 1 row affected
  > select * from a where x = 0:
  +---+---+
  | x | y |
  +---+---+
  | 0 | 1 |
  | 2 | 3 |
  +---+---+

  i.e. we were matching (1,0) and then trying to delete (0,1).
  This fixes the intermediate row key tuple to match the pk order. Test in dolthub/go-mysql-server#854.
• 2940: Grammar fixes
• 2938: go/libraries/doltcore/sqle: Remove some dependencies on pkg types
  This PR
  • refactors the dolt_schemas and dolt_procedures tables to remove direct types.Map dependencies.
  • refactors table to remove direct types.Map dependencies.
  • modifies DoltIndex to only push down filters for the current format, not for prolly indexes.
  • skips unsupported engine tests for the new format so that the engine suite can be run without panicking
• 2930: Prevent Multiple Not-NULL Constraints
  Removed code where we would add an additional NOT NULL constraint when dropping primary keys.
  Should be impossible to add multiple NOT NULL constraints to a single column, as there are checks right before anywhere we append a NOT NULL for if there already is NOT NULL constraint.
  Added a method in schema marshalling that'll remove all duplicate not null constraints.
• 2929: go/store/datas: Begin work for moving top-of-DAG chunks to flatbuffers messages.
  This starts on the changes necessary to implement __DOLT_1__ format with flatbuffers messages representing top-of-DAG entities like StoreRoot, WorkingSet, Commit, Tag, RootValue, etc. This particular PR creates machinery to move StoreRoot, WorkingSet and Tag to simple flatbuffer message representations. This PR includes:
  • Move CommitMeta, TagMeta and WorkingSetMeta to store/datas package.
  • Move accessors in doltcore/doltdb of WorkingSet and Tag heads to use new HeadTag and HeadWorkingSet methods on the Dataset interface instead.
  • Add a new __DOLT_DEV__ format which uses these top-of-DAG flatbuffer messages.
  • Add serial.StoreRoot message which stores an un-chunked RefMap of roots.
  • Add serial.Tag message which stores the tag metadata and the hash of the commit.
  • Add serial.WorkingSet message which stores the working set metadata, the hash of the current root value, hash of the staged root value and hash of the merge state.
  • Add a types.SerialMessageKind and types.SerialMessage type, which is implemented like InlineBlob but can also introspect the serialized message to WalkRefs the addresses of it.
• 2924: Change HedgeStrategy interface to control nextTry durations directly
  Strategy has been renamed to HedgeStrategy. HedgeStrategy's now control the nextTry durations directly and an ExponentialHedgeStrategy has been added that can be composed with other strategies to get the previous behavior.

go-mysql-server

• 862: fix misspelled name in information_schema.innodb table
• 861: have methods for dolt to call to parse check expression
  To properly merge check constraints, Dolt needs to partially parse their expression strings.
  This PR gives Dolt access to some helper methods, to complete this operation. Should have no functional effect on GMS.
• 860: Add all non INNODB information_schema tables.
  This PR adds a bunch of information_schema tables that were previously missing. All are initialized with an empty row iter.
• 859: add all information_schema.innodb_* tables as empty tables
• 856: Only skip updating defaults when added column has no default value AND is nullable
  Fix for bad PR #855 which fails the following Dolt test:
  http://github.com/dolthub/dolt/blob/51987ab84ef9d07b3c6ceae3ab65b9ca65ab55bd/go/libraries/doltcore/sqle/sqlddl_test.go#L482-L482
  Verified this PR passes Dolt tests.
  Also, removes TestAddAndDropColumn test which will be replaced with an appropriate bat test on Dolt.
  Related to #2939
• 855: Don't apply defaults to all rows when the added column has no default
  Related to #2939
  When a user was working with a large table, adding a column with no default caused an OOM crash. The crash is a result of the ADD COLUMN sql statement causing every row to be modified in the table. This is not-expected and a bug. We expect adding a column with no default to only modify the schema and not any Dolt row.
  The fix is to skip applying defaults to table if the added column has no default. We can call this a "fast" add column since only the schema has to be modified.
  In the future, we may also add a "fast" drop column where only the schema is modified. Currently, we modify every row for a DROP COLUMN statement. In the "fast" drop scenario, it might be important to ensure values aren't retained across ADD COLUMN and DROP COLUMN statements with the same column name. This PR also adds tests to ensure this.
• 854: Pk ordinals delete test
• 853: removing routine_definition column from information_schema.routines table in enginetest as it gives different values in GMS and Dolt
  The value in routine_definition comes out different:
  • in Dolt, Exchange(parallelism=3)\n └─ Project(6)\n └─ Table(dual)\n
  • in GMS, Project(6)\n └─ Table(dual)\n
• 852: add information_schema.routines table
• 850: Added sql.FilteredIndex to allow integrators to specify index filter pushdowns
• 848: Add FLUSH PRIVILEGES support

vitess

• 133: reserve found and allow as column safe
• 132: Add FLUSH as a command and its options

Closed Issues

• 2960: Runtime error: invalid memory address or nil pointer dereference error when cloning a Dolt repository
• 2859: Support FLUSH PRIVILEGES syntax
• 2949: dolt table import cannot handle table names with - in them
• 2879: SELECT 1 as found breaks
• 2462: Support -a in dolt tag command
• 2858: Implement the information_schema.routines table
• 2927: Columns can have more than one not-null constraint in storage
• 2939: Error altering table - high memory usage and process is killed
• 2919: 3-way schema merge creates duplicate columns
• 2890: Keyless Tables Do Not Support Rollback Semantics
• 1390: document the format of s3 remote strings