Merged PRs

dolt

• 5038: Block merges that corrupt indexes
  Today, many merges in the new format can leave indexes in a corrupted state. This PR blocks those merges as a short-term fix. In the long term we will make these merges automatic.
• 5030: go/store/nbs: Update chunk journal to support GC
• 5025: Fix instructions for first connect
  Fixes #5024
• 5018: go/store/{blobstore,nbs}: Added conjoin for GCS remotes
• 5017: Sysbench summaries for testing
• 5012: Move TableFileStore from nbs to chunks
• 5006: Rewrite broken-merge.bats in enginetest package
  Rewrite of broken-merge.bats in engine test package. Adds additional tests.
• 5005: Perf improvements for remote read replicas
  This significantly speeds up the process of syncing a remote-based replica by 1) pulling chunks once, rather than once per branch in the case of replicas that want everything, and 2) by eliminating a bunch of duplicated and wasted work. Latency for up-to-date case is now <100 ms.
• 5004: Split dolt_queries into three files.
• 4997: go/store/prolly: Refactored prolly diff iter to accept zero-value maps
• 4994: Fix remotes-aws.bats for DOLT
• 4985: adding tests for prepared, execute, and deallocate
• 4976: Fixed DELIMITER
  Previously we required delimiters to have the previous delimiter at the end, but that requirement no longer exists, which matches MySQL's behavior.
• 4956: Batch replication branch pulling

go-mysql-server

• 1496: skip once after for stored procedures
• 1494: add 'privileges' column for information_schema.columns table
  Made information_schema database and its tables accessible to all users.
  Note:
  • column privileges is not supported yet.
  • view columns not supported yet.
• 1493: cte with same name in subquery
  fix for: #4950
• 1491: Revert "resolve cte before resolving tables"
  Reverts dolthub/go-mysql-server#1485
• 1490: /sql/datetimetype: fix ranges to match mysql
• 1486: allow naked identifiers for as of statements
  added a special case for unresolved columns for statements using as of <identifer>
  fix for: #3116
• 1485: resolve cte before resolving tables
  Changes:
  • resolve the CTE before validating procedures
  • handle SubqueryAlias
    Fixes: #4480
• 1481: Truncate now checks system variable foreign_key_checks
  Truncate now checks system variable foreign_key_checks and does not throw if the variable is not 1.
  This fixes #5007
• 1480: add some supported columns for STATISTICS and COLUMNS info schema tables
  • Added sub_part for STATISTICS table.
  • Fixed column_default, datetime_precision, extra and other columns for COLUMNS table.
  • Skipped cardinality column for STATISTICS table as it's not supported in Dolt yet.
  • Skipped privileges column for COLUMNS table for now. (in progress)
  • Modified all Dolt supported MySQL functions to use its FunctionName() method to create the string statement of the function itself. The FunctionName() method returns the name of the function in all lower cases. This makes it so that all Dolt supported MySQL functions are consistent to what MySQL returns for queries such as SHOW CREATE TABLE or the column_default of information_schema.COLUMNS table.
    Notes:
  • Dolt does not support Time length yet.
  • Dolt currently handles BOOL and TINYINT data types the same with INT8, but MySQL explicitly defines BOOL as tinyint(1), which is different from tinyint for TINYINT data types.
  • View columns are not available in COLUMNS table yet.
• 1479: check that all bindings are used
  Fix for: #4481
• 1477: added collation support for STRCMP
  adding collation support for STRCMP
• 1476: group_by: collation-aware grouping
  Hi doltfriends! Here's another compatibility fix. This one I'm more confident about: GROUP BY is not collation aware when grouping!
  The included test reproduces the issue trivially, but for completeness: when using a GROUP BY statement that includes a textual column, the column's collation needs to be taken into account when creating the aggregation groups.
  E.g., for the following schema and query

  CREATE TABLE test (id bigint, title varchar(255) collate utf8mb4_0900_ai_ci, PRIMARY KEY(id));
  INSERT INTO `test` (title) VALUES ('val1'), ('VAL1'), ('val2'), ('VAL2')
  SELECT COUNT(*) FROM test GROUP BY title

  We should expect the result of the SELECT to contain two rows, as there are only two groups when grouping by title. Even though we've inserted 4 unique values, the utf8mb4_0900_ai_ci with which the column is collated is case insensitive, so in practice, 'val1' = 'VAL1' and 'val2' = 'VAL2' when grouping. This is not the current behavior.
  The fix is straightforward: use a weight string when hashing to generate the grouping key instead of using the expression's literal value. I did a small refactoring in collations.go so that we can write the weight string directly into the grouping key's hasher and continue using the same code for the existing HashUint API.
  cc @dbussink
• 1475: Fixed stored procedure panics
  This fixes two panics that were discovered for stored procedures. The first comes from this issue: #4980. The second deals with an issue with external stored procedures, where the usage of an uninitialized user variable would throw a panic over the nil value.
• 1470: Subquery caching rules onceAfterAll
  We cannot cache subquery results for a scope until the child scope has been finalized. A child scope cannot be finalized until the parent scope is finalized. Caching subqueries has to happen after all scopes are finalized. Applying hash lookups in the old way is much more difficult with this setup. This also exposed some bugs related to cacheability checks and subquery/union finalization recursion. Rewrite rules that depend on subquery caching to be in order traversals in OnceAfterAll.
  The original bug sourced from a bad resolve in a query like this:

  select ab.* from ab join pq on a = p where b = (select y from xy where y in (select v from uv where v = b)) order by a;

  The join and two scopes appear necessary to trigger the original bug. The join's children will be unresolved when we index the inner scopes, whereas a single relation scope's child will be a resolved table. The two scopes provide a gap for determining cachability in the middle scope while the child scope still had a deferredColumn, which previously permitted caching.
  GMS bump: #4987
• 1448: implementing PREPARE, EXECUTE, and DEALLOCATE
  fx for: #4452

Closed Issues

• 5035: Inaccurate ranges for DATETIME and TIMESTAMP
• 4950: Multiple CTEs with the same name can cause incorrect results
• 4480: Analyzer bug when using CTEs in stored procedures
• 3116: Allow AS OF strings to be naked identifers
• 5024: README instruction does not work for connecting
• 4481: Unused query bindings should be an error
• 5007: TRUNCATE with foreign keys doesn't always work
• 5000: unexpected dolt sql console crash
• 4452: Support prepared statements SQL statements
• 4982: query could not complete when rows are done
• 3253: support executing delimiter statement without requiring query terminator
• 4975: Intermittently receive the message error reading from or writing to noms: new manifest created with non 0 lock when running dolt in docker.
• 4320: call dolt_backup() doesn't work if called on a sql-server started in a parent directory.
• 4977: Can't create database with navicat: Dolt does not accept single quotes on charset identifier

Latency