This is a patch release containing new features and bug fixes.

Features:

A new system table function, dolt_diff(). Read more about it here
SHOW CREATE PROCEDURE is now supported

Bugs fixes:

Error in connecting to a commit hash revision when in read replica mode
Incorrect handling of foreign key creation on tables with spatial columns
Incorrect handling of foreign key creation for tables with multi-column primary keys
CHARACTER SET tokens in CREATE TABLE statements now parse correctly
Incorrectly dropped WHERE clauses in certain indexed joins
Various information_schema tables now contain more accurate information
Columns and table names that begin with numbers but have at least one letter (123a) are now valid

Merged PRs

dolt

3055: Bug fix for connecting to a particular commit hash in a read replica
Fixes #3056
3054: /{.github,go}: require go 1.18
3053: Simplify RebuildIndex
Two improvements with index rewriting:
1. Brian's TupleFactory optimizations were being skipped, and 70% of
  the runtime was spent building newBinaryNomsWriter instances for tuple
  construction. Removing this restores index inserts as the only memory
  pressure.
2. Brian's BulkEditAccumulator lets the final Map sort skip
  work materializing intermediate prolly trees.
  On a 2000 row table, fix (1) reduces the runtime by 50% (80ms -> 40ms),
  and fix (2) reduces the runtime by another 40% (40ms -> 10ms)%, for a
  cumulative ~90% improvement.
3042: Fixed two bugs in foreign key creation
Fixed #3023
Fixes #3028
3041: go/cmd/dolt: Fix fetch printing newline for each progress update on Windows
Fixes: #2941
The above reported bug is caused by EphemeralPrinter. It's related to how the underlying library of uilive clears terminal lines on Windows. uilive will only clear terminal lines successfully if it detects that the given io.Writer is a terminal. The package does this by checking if the io.Writer has a Fd function and using the returned file descriptor for its istty check.
When color.Output is used as the io.Writer, uilive's terminal check fails since that does not declare Fd. The problem is that color.Output does use the initial os.StdOut which is in fact the terminal...
This was fixed by wrapping color.Output with a struct that reports the correct file descriptor.
This PR also improves the display output of ephemeral printer by controlling when uilive flushes its output.
3038: go.mod,proto/third_party: Bump protobuf dependencies to migrate from github.com/golang/protobuf to google.golang.org/protobuf.
3036: unskip bats test for CREATE DB ... CHARACTER SET ... statement
3035: go/libraries/sqle: Added integration tests for "Database Revisions", simplified DoltDatabseProvider
3030: Unblock the failing MySQL FDW integration test
This pr:
1. Switches to an older version of MySQL FDW due to #3029
2. Adds a comment in the c api to test an additional function
3. Adds logic in the mysql bats client tests that shuts the down the postgres server in the teardown. Will be useful in case new versions of the wrapper cause another breakage.
3025: .github/workflows: Added timeout for MySQL client integration tests
3021: go/libraries/doltcore/sqle/dsess: Refactor Dolt Session update methods
This PR alters the life cycle of WriteSessions within a dolt session. WriteSessions now have a single working set ref throughout their existence, and assert that their working set ref is not changed by set or update calls.
To switch branches/working set, the Session discards the existing (empty) WriteSession, and creates a new one

2913: dolt_diff system table function support
Adds support for a new dolt_diff system table function and changes the behavior of the existing dolt_commit_diff_$tablename and dolt_history_$tablename system tables to no longer disambiguate column names with tag suffixes.
The new dolt_diff system table function enables callers to use the exact to and from schemas in a two-way diff of table data. Using this sample data input, the following examples show what the dolt_diff table function returns in different cases.
General example:

> select * from dolt_diff("foo", @Commit1, @Commit9);
+------+------+-------+----------------------------------+-----------------------------------+--------+--------+---------+----------------------------------+-----------------------------------+-----------+
| to_c | to_b | to_pk | to_commit                        | to_commit_date                    | from_a | from_b | from_pk | from_commit                      | from_commit_date                  | diff_type |
+------+------+-------+----------------------------------+-----------------------------------+--------+--------+---------+----------------------------------+-----------------------------------+-----------+
| jkl  | foo  | 1     | dap9j5he1296trqe5dp4j2n9lut9tuaa | 2022-03-16 20:13:05.878 +0000 UTC | foo    | bar    | 1       | diq7v3vlqpjb21imb3bqv04spob7l0p2 | 2022-03-16 20:13:05.814 +0000 UTC | modified  |
| NULL | baz  | 2     | dap9j5he1296trqe5dp4j2n9lut9tuaa | 2022-03-16 20:13:05.878 +0000 UTC | baz    | bash   | 2       | diq7v3vlqpjb21imb3bqv04spob7l0p2 | 2022-03-16 20:13:05.814 +0000 UTC | modified  |
| five | four | 3     | dap9j5he1296trqe5dp4j2n9lut9tuaa | 2022-03-16 20:13:05.878 +0000 UTC | NULL   | NULL   | NULL    | diq7v3vlqpjb21imb3bqv04spob7l0p2 | 2022-03-16 20:13:05.814 +0000 UTC | added     |
+------+------+-------+----------------------------------+-----------------------------------+--------+--------+---------+----------------------------------+-----------------------------------+-----------+

Diff across a column drop:

> select * from dolt_diff("foo", @Commit1, @Commit2);
+------+-------+----------------------------------+-----------------------------------+--------+--------+---------+----------------------------------+-----------------------------------+-----------+
| to_a | to_pk | to_commit                        | to_commit_date                    | from_a | from_b | from_pk | from_commit                      | from_commit_date                  | diff_type |
+------+-------+----------------------------------+-----------------------------------+--------+--------+---------+----------------------------------+-----------------------------------+-----------+
| foo  | 1     | frfpd9k00s417gi8b88067l5oavkfpt5 | 2022-03-16 20:13:05.821 +0000 UTC | foo    | bar    | 1       | diq7v3vlqpjb21imb3bqv04spob7l0p2 | 2022-03-16 20:13:05.814 +0000 UTC | modified  |
| baz  | 2     | frfpd9k00s417gi8b88067l5oavkfpt5 | 2022-03-16 20:13:05.821 +0000 UTC | baz    | bash   | 2       | diq7v3vlqpjb21imb3bqv04spob7l0p2 | 2022-03-16 20:13:05.814 +0000 UTC | modified  |
+------+-------+----------------------------------+-----------------------------------+--------+--------+---------+----------------------------------+-----------------------------------+-----------+

Diff across a column drop and rename:

> select * from dolt_diff("foo", @Commit1, @Commit3);
+------+-------+----------------------------------+----------------------------------+--------+--------+---------+----------------------------------+-----------------------------------+-----------+
| to_b | to_pk | to_commit                        | to_commit_date                   | from_a | from_b | from_pk | from_commit                      | from_commit_date                  | diff_type |
+------+-------+----------------------------------+----------------------------------+--------+--------+---------+----------------------------------+-----------------------------------+-----------+
| foo  | 1     | tnipgs33r3kd0pt823u0uu5ek4n39dpf | 2022-03-16 20:13:05.83 +0000 UTC | foo    | bar    | 1       | diq7v3vlqpjb21imb3bqv04spob7l0p2 | 2022-03-16 20:13:05.814 +0000 UTC | modified  |
| baz  | 2     | tnipgs33r3kd0pt823u0uu5ek4n39dpf | 2022-03-16 20:13:05.83 +0000 UTC | baz    | bash   | 2       | diq7v3vlqpjb21imb3bqv04spob7l0p2 | 2022-03-16 20:13:05.814 +0000 UTC | modified  |
+------+-------+----------------------------------+----------------------------------+--------+--------+---------+----------------------------------+-----------------------------------+-----------+

Depends on:

go-mysql-server

904: IndexedJoins mark handled filters
Applying an indexed join in applyJoinPlan should apply equality
lookup expressions, but currently do not. pushdown mistakenly
marked those expressions as handled and eliminated necessary filters.
This PR prevents IndexedJoinAccess with nil lookups from marking
filters as handled.
A future improvement could the pushdown range logic
to opportunistically apply lookups in applyJoinPlan, and let
pushdown mark those as handled. This would improve perf by both
removing both a table scan and filter from some index joins.
899: run query on info schema routines table for SHOW PROCEDURE STATUS
Also fixes SHOW PROCEDURE STATUS to show all procedures in all databases.
Removes show_procedure_status.go file (removing the showProcedureStatus node)
897: Code sample update to remove missing Auth package
Updated code sample in README and _example/main.go to remove the missing Auth package.
Reported in: dolthub/go-mysql-server#888
896: adding 1 second to trigger creation time
Fix for this issue: #3047
Triggers weren't showing up in TablePlus because the creation time for triggers is always 0 in unix time, which is out of TablePlus supported timestamp range (for some reason). So this PR just makes the trigger creation time 1 second after the start of all time, which TablePlus considers valid.
895: fix SHOW INDEXES FROM otherdb.tab fails as database is initialized incorrectly in parsing
Before Show Index statement was initializing the Database incorrectly as it would set it to either empty or current database. This was an issue for using the statement for database that user is not currently on.
This PR also fixes couple of info schema table inconsistencies against Mysql.
893: /go.mod: require go 1.18
887: fix non_unique column value definition in information_schema.statistics table
fix non_unique being assigned incorrect value. Verified with Mysql docs, "0 if the index cannot contain duplicates (which .IsUnique() is true), 1 if it can."
885: Ensure CheckPrivileges runs on resolved table functions
While testing privilege checks for the new dolt_diff table function, I realized that CheckPrivileges was being run on UnresolvedTableFunction and not the actual resolved table function. Since the dolt_diff table function pulls data from underlying tables, it needs to be resolved before CheckPrivileges runs to ensure we can accurately check privileges.
884: added warnings for some syntaxes of ALTER TABLE and CREATE DATABASE statements
- Added a warning for using CHARACTER SET, COLLATE or ENCRYPTION syntaxes when creating database. The database is still created.
- Added a warning for using DISABLE KEYS or ENABLE KEYS options for alter table statement. Nothing changes for the table being altered.
- Added a system variable that mysqldump uses, innodb_stats_auto_recalc
881: More tests for alter table bugs
1. Split PkOrdinals tests into DDL and DML (new)
2. New script test that triggers the primary key / secondary key ordinal mismatch
3. Add ctx.Done() checks to a few key node iterators before operating on child rows (may have missed some, but tried to hit the table, index, and edit iterators)
  companion PR: #3013

vitess

147: Have SHOW CREATE PROCEDURE <procedure_name> parse
TODO: need changes in GMS to handle show create procedure logic.
146: /go.mod: require go 1.18
143: Allow Columns to Start with Numbers...kinda
Recognizes identifiers that start with numbers, unless they are immediately after a .
TODO: table names that start with numbers do parse, but there is a regex blocks these from parsing in dolt, so will fix in future PR.
142: Add DISABLE | ENABLE KEYS syntax for ALTER TABLE statement
Added parsing of DISABLE KEYS and ENABLE KEYS options for ALTER TABLE statement
Updated checking Index action keyword to be case insensitive
140: support CHARSET, COLLATE and ENCRYPTION syntax in CREATE DATABASE statement
added parsing of CHARACTER SET, COLLATE and ENCRYPTION options for CREATE DATABASE statement
139: Revert "Merge pull request #137 from dolthub/james/special-comments"
This reverts commit 19d17da876ea71be47f41092d8ab420885b9ba18, reversing
changes made to 0bb5262822024a793f65040a77430ebe0624058e.

Closed Issues

3044: Renaming column fails on TablePlus as it generates query with DEFAULT value specified.
2982: Cannot add a column that starts with a number
3056: use db/<hash> breaks when server replication enabled
2941: Dolt pull/push progress doesn't clear last reported progress on Windows
3028: UNIQUE constraint inappropriately created during foreign key creation
3023: Foreign key on a table with spatial type column triggers a spurious error cannot create an index over spatial type columns
2463: Missing features in dolt log
2971: CHARACTER SET in CREATE DATABASE statement results in SQL parsing error
888: The _example/main.go compile error.

dolthub/dolt v0.37.7 0.37.7 on GitHub

Merged PRs

dolt

go-mysql-server

vitess

Closed Issues

dolthub/dolt v0.37.7
0.37.7

on GitHub