This is a patch release, containing minor features, bug fixes and performance improvements.

It adds the following features:

dolt_diff system table now indicates whether a table in a diff had data changes, schema changes, or both
dolt_diff_table_name system tables have simpler schemas in the presence of historical schema changes to the table
SHOW STATUS LIKE now parses
Faster table scans when using an index
Named windows can now be used

It addresses the following bugs:

Slow foreign key check logic when no value was changed
NULL values in dolt dump
Duplicate indexes can now be created
Clone and pull commands now correctly use "downloading" language, better status output
Bug fix for BETWEEN expressions with a NULL bound
DDL statements that failed when no database was selected now work

Merged PRs

dolt

2888: don't loop through all fks, if change doesn't impact any fk columns
2875: fix null value being exported for any types into .sql file
the issue covered all typeInfo types where null values were not getting exported correctly.
the fix is to add NULL whenever the value is nil and cannot be converted into sql types.
2867: go/store/nbs: Add OverwriteStoreManifest to nbs package
In some cases, it may be desirable to add tables files to the manifest without opening the new table files as chunkSources. For large clone operations, it can be problematic to load the table file chunkSources as this causes their indices to be loaded into memory. In order to reduce the memory overhead of a clone, this PR adds OverwriteStoreManifest.
OverwriteStoreManifest adds table files to the manifest file and persists it. It will not call Rebase, unlike UpdateManifest.
Care should be taken when using OverwriteStoreManifest as old table files will not be garbage collected. Overwriting the manifest multiple times may allow the store to grow arbitrarily large in size.
2864: doltcore/sqle: Optimize Delete
Fetching and updating auto increment values is relatively expensive given the current implementation. Conditionally skipping this step makes deleted 2x faster
2857: Removing use of SuperSchema from Dolt_Blame system view.
Quick change to remove the SuperSchema reference from Dolt_Blame system view, now that we have migrated Dolt_DIff_$tablename off of SuperSchema's tag-suffixed column names.
2851: libraries/utils/editor: fix dropped error
This fixes a dropped err variable in libraries/utils/editor.
2848: unskip dropTable bats test
2844: go/store/prolly: Faster table scans

2843: Expand DOLT_DIFF system table to indicate the type of change
This change adds two new boolean columns to the unscoped DOLT_DIFF system table to enable customers to determine if a change to a table was a schema change or a data change (or both).
Example Usage:

> create table x (a int primary key, b int, c int);
> create table y (a int primary key, b int, c int);
> insert into x values (1, 2, 3), (2, 3, 4);
> select DOLT_COMMIT('-am', 'Creating tables x and y');
> select * from dolt_diff;
+----------------------------------+------------+-----------+-------------------------+----------------------------------+-------------------------+-------------+---------------+
| commit_hash                      | table_name | committer | email                   | date                             | message                 | data_change | schema_change |
+----------------------------------+------------+-----------+-------------------------+----------------------------------+-------------------------+-------------+---------------+
| 1blnkur3m1hla2a4got9t513982a4ad1 | x          | jfulghum  | jason.fulghum@gmail.com | 2022-02-21 14:39:37.01 -0800 PST | Creating tables x and y | true        | true          |
| 1blnkur3m1hla2a4got9t513982a4ad1 | y          | jfulghum  | jason.fulghum@gmail.com | 2022-02-21 14:39:37.01 -0800 PST | Creating tables x and y | false       | true          |
+----------------------------------+------------+-----------+-------------------------+----------------------------------+-------------------------+-------------+---------------+

Resolves: #2834

2840: go/dolt/cmd: Use download language when dolt pulls from a remote
Fixes #2798
For downloads (pulling chunks from a remote db to a local db), dolt will now report the number of tables files written to disk as Files Written. It will no longer report an Upload Rate for a download as that is the speed at which the file is copied from a temp directory.
For uploads (pulling chunks from a local db to a remote db), dolt will now report the number of table files created during the pull process as Files Created and the number of these tables files that have been uploaded as Files Uploaded.
2835: go/store/val: Remove NULL bitmask from new tuple format
2833: store/types: fix dropped error
This fixes a dropped err variable in the store/types package.
2832: Simplify DOLT_DIFF_$TABLE system tables by removing tag suffixes
When the DOLT_DIFF_$TABLE system tables show schema history with column name conflicts, the column names are disambiguated by adding their unique tags as suffixes. This makes it difficult to work with these system tables. This change simplifies the output by restricting the output schema to be based on the current table schema and avoids any column name conflicts.
Resolves: #1910 (for dolt_diff_$table, but does not change `dolt_commit_diff_$table yet)
2831: go/store/val: Added fast field access for fixed size tuple fields
2829: unskip fixed tests
- un-skipped some tests that are resolved
- fixed some test typos
- added test for add foreign key constraint bug
2828: Remove index filters
2826: Add the ignore skipped row param
Adds a flag to skip rows that are printed by --continue.
2825: Removing Duplicate Index Constraint
2822: go/cmd/dolt: Improve progress reporting of clone
Previously, it was difficult to understand the progress of dolt clone, especially when you are cloning a db with a small number of large table files. dolt clone now lists which table files are being concurrently downloaded and shows the progress and download rate for each.
An example:

@reltuk Brought up the concern for compatibility between the new uilive dep and github.com/faith/color. In my testing there doesn't seem to be any issues:
2812: Modify Benchmark Configuration for End to End support of the TPCC benchmark
2789: fix dolt log pager panics with ctrl+c on Windows
added catching Interrupts for dolt log as it panics with Ctrl+C on Windows with more command

go-mysql-server

827: fix ROLLBACK; fail with no database selected
MySQL lets ROLLBACK when no database is selected
826: Don't call ProcessList.Update on every row spooled in a result set by default
824: analyzer doesn't error for invalid value
re: dolthub/go-mysql-server#815
Originally we were failing to handle an analyzer error. Catching that error exposed a case where we want to manually prevent that error path from bubbling up.

820: Add uint types to LiteralToInt
Re: issue dolthub/go-mysql-server#799
This database and this query were parsing 128 as a uint8, which my type switch missed:

$ dolt clone https://doltremoteapi.dolthub.com/post-no-preference/stocks
$ cd stocks
$ dolt sql
stocks> select date, act_symbol, avg(close) OVER (PARTITION BY act_symbol ORDER BY date ROWS BETWEEN 128 PRECEDING AND CURRENT ROW) AS ma200 FROM ohlcv WHERE act_symbol='AAPL' having date = '2022-02-11';
offset must be a non-negative integer; found: 128

After fix:

select date, act_symbol, avg(close) OVER (PARTITION BY act_symbol ORDER BY date ROWS BETWEEN 128 PRECEDING AND CURRENT ROW) AS ma200 FROM ohlcv WHERE act_symbol='AAPL' having date = '2022-02-11';
+-------------------------------+------------+--------------------+
| date                          | act_symbol | ma200              |
+-------------------------------+------------+--------------------+
| 2022-02-11 00:00:00 +0000 UTC | AAPL       | 158.29837209302272 |
+-------------------------------+------------+--------------------+

I haven't been able to create a testing database with the same type parsing behavior yet. Something about the stocks database or that specific query is yielding a types.Value with value=128 and type=sql.Uint8.

819: fix BETWEEN function of missing inclusive logic
BETWEEN() does inclusive check including checks with NULL value
817: Changing UserVar to return its correct type
Changing UserVar to return its correct type, when used as a value in an expression.
Resolves: dolthub/go-mysql-server#790
816: /go.{mod,sum}: bump vitess
813: /sql/analyzer: Added feature flag for single threaded execution
811: fix DropTable enginetest
fix DropTableTest in enginetest, was causing failed test in Dolt
810: Extending script queries to allow easier testing of warning messages
This change lets you specify the expected number of warnings and an expected substring from test queries. It also tidies up some of the logic around running tests with expected warnings.
Example test syntax this allows you to write:
```
Assertions: []enginetest.ScriptTestAssertion{
{
Query:                           "select * from dolt_diff_t;",
ExpectedWarning:                 1105,
ExpectedWarningsCount:           4,
ExpectedWarningMessageSubstring: "unable to coerce value from field",
SkipResultsCheck:			true,
},
},
```
Needed for: #2832
809: query plan test for duplicate indexes
806: Deduplicate IndexLookup filters
Opportunistically eliminate filter expressions when
sql.IndexedTableAccess is used for a table scan. The original logic
respecting join scoping for applying indexes is unaffected, this simply
connects out current filter elimination logic with the expressions used
for range scan.
This requires integrators implementing sql.IndexLookup to only return
rows selected by the lookup filter.
Changes were made to ranges and datetime filters To allow integrators
to comply to this filtering standard. GMS can now distinguish between
NULL and infinity. Tests are added for IS NULL and IS NOT NULL handling
for single and multi column expressions. Engine datetimes are not truncated
to support date index lookups with the same precision as MySQL.
We currently fail to eliminate filter inversions. More work is needed
to compare inverted filters to their range scan representation.
804: Revert "addColumn"
This reverts commit f64b71875501714d69ed9b80c38e2564467d2794.
803: Add NamedWindows Node
This moves window name validation, resolving, and embedding out of
parsing and into the analyzer.
802: Add vitess to go mod to support SHOW STATUS LIKE
801: Added external functions for persisting Grant Tables
This adds the functions necessary to be able to easily persist the data from Grant Tables as JSON for an integrator.
794: Fix drop table from different database
Added DropTable case in getTransactionDatabase
Added tests to check dropping table in out of scope database
Changed ddlNode to CurDatabase that is update with every table being dropped
Changed taking array of tables as string type to plan.UnresolvedTable type
Updated resolve-table rule in analyzer to support DropTable filtering out non-existent tables.
789: fix alter table columns in different database
Removed ddlNode and added tests for AddColumn , DropColumn , RenameColumn , DropColumn that check updates with different database tables than current one.
Added cases in engine.go for updating the transaction database for some plans including above and CreateIndex , AlterIndex

vitess

126: go/mysql: Add buffering to prepared statement result sets
121: Add show like status
120: Named Windows
Add yacc syntax for window names. WindowDef is used with either its Name
field in a WINDOW clause, or NameRef in an OVER clause. NameRef is OVER
is not mandatory, but Name in WINDOW is.
This PR adds precedence rules to avoid shift/reduce conflicts between
the WINDOW name field, which can be any string, and RANGE, PARTITION,
ROWS, and RANGE should be prioritized for other rules if they appear as
the first token in a window_spec.

Closed Issues

2860: Bad error with rollback on unselected database
2874: dolt dump - Error with dumping
2855: Panic on information_schema.tables
2674: UPDATE in trigger statements don't work with multiple rows
2729: Allow duplicate indexes to be created
2834: Expand DOLT_DIFF system table to indicate the type of change
2824: Unexpected --continue behaviour for dolt table import -u
2783: Support `SHOW SESSION STATUS LIKE...' syntax.
2798: Dolt fetch, pull, etc. use "upload" language
1910: Schema changes for dolt_diff_$tablename and dolt_commit_diff_$tablename
2779: Panic on SIGINT during commit log
2598: Dolt ignores schema name in DROP statements
2764: Some DDL statements can't be chained in a single session but work when applied in multiple sessions
2821: New flag to suppress skipped rows when using --continue flag with dolt table import
2753: cli arguments should optionally override config values
2802: Every SQL Commit is a Dolt Commit
815: Index error lost in parent call
790: String variables aren't evaluated correctly in where clauses
799: Aggregate Partition Window Rows beyond 127
146: regex-metrics can't be initialized
171: Two foreign keys in same CREATE TABLE statements triggers error
355: Question/Feature Request: How can I increase the parallelism of expression evaluation?
654: fatal error: concurrent map write read
705: error: can't modify database databaseProvider
709: create table with unnamed key got error ERROR 1105 (HY000): Error: index already exists
191: Support for VALUES in INSERT ... ON DUPLICATE KEY UPDATE
250: unsupported syntax: create database mapping
313: How can I custom my log.
481: Anyone working on the replication?

dolthub/dolt v0.37.2 0.37.2 on GitHub

Merged PRs

dolt

go-mysql-server

vitess

Closed Issues

dolthub/dolt v0.37.2
0.37.2

on GitHub