Merged PRs
dolt
- 5280: go/store/datas/pull: trim memory footprint for puller hash sets
- 5275: verify fk with secondary index even when empty
The optimization for verifying foreign keys using secondary indexes isn't used when the tables are completely different (so only additions to the merge) or if the index potentially didn't exist before merging. Rather than defaulting to using the primary key to verify the foreign keys, it's possible to just create an empty index and use the faster secondary index to verify.
This includes code from #5253 - 5253: verify fk with secondary when available
We resolve foreign key conflicts by:- diffing a before/after index to generate keys
- constructing a lookup key for the reference index
- perform a lookup into the reference index
- report a conflict if reference dependency is missing
For huge tables, we want thefrom
keys to be ordered relative to theto
index. When this is not the case, theto
lookup will be random access and read a chunk from disk. Millions of random access lookups that all IO chunks from disk adds a ~1000x perf hit for validating a single FK relative to sorted lookups.
This PR does a secondary key diff -> secondary foreign key lookup. The diffs are ordered relative to one-another, and this usually makes big merges faster.
The main downside is that constraint violations still require a primary key lookup to backfill missing columns, which will usually be out-of-order relative to the secondary key iter order. Keyless fks are also excluded from the optimization for now.
go-mysql-server
- 1574: Subquery caching correctness
Rerunning cacheSubqueries recursively on subquery expressions led to an incorrect cacheability labelling. The rule should only be run once on a top-level tree. - 1573: add support for
SHOW GLOBAL VARIABLES
- 1570: Fix panic by rewriting how we move filter conditions up.
Fix panic by rewriting how we move filter conditions up.
Fixes #5214
vitess
- 218: Fixed bug in COM_FIELD_LIST command implementation for table names that need quoting in a statement
Fixes #5151
Fixes #4464 - 217: Adding implicit string concatenation for a variable assignment needed for binlog replication
When amysqldump
is taken for a database tracking GTIDs, it outputs GTID tracking information via a line like:In MySQL 8.0+, the MySQL comment inserts aSET @@GLOBAL.GTID_PURGED= /*!80000 '+'*/ 'beabe64c-9dc6-11ed-8021-a0f9021e8e70:1-126';
'+'
between the=
and the assignment value, which gets implicitly concatenated with the adjacent quoted string in MySQL.
GMS/Dolt doesn't support this implicit string concatenation, but I wanted to get this query to work – otherwise customers will have to manually edit their mysql dump files to remove this syntax before it can be imported into Dolt. I didn't think it was worth taking on the larger change right now to support implicit quoted string concatenation everywhere (#5232), so instead, I added special handling for this case since it's important for the binlog setup experience. - 216: Fix int overflow on 32-bit architectures
$ GOARCH=386 GOOS=linux go build ./... # github.com/dolthub/vitess/go/bucketpool go/bucketpool/bucketpool.go:57:15: math.MaxUint32 (untyped int constant 4294967295) overflows int
- 215: Adding two new binlog replication statements:
RESET REPLICA
andCHANGE REPLICATION FILTERS
Closed Issues
- 5151: mysql client's
information_schema
read with table names containing-
can put the client in a bad state. - 4464: MySQL client with autocomplete enabled crashes connecting to Dolt if a table is named a reserved word like
rows
- 5214: dolt panic during JOIN
- 5175: High load in 1) SELECT queries when we have 7000 rows and 21000 branches 2) create extra branches when we've already have 21000
- 5080: Can't edit table's data in navicat
Latency
Read Tests | MySQL | Dolt | Multiple |
---|---|---|---|
covering_index_scan | 2.0 | 2.81 | 1.4 |
groupby_scan | 12.75 | 16.71 | 1.3 |
index_join | 1.16 | 4.57 | 3.9 |
index_join_scan | 1.14 | 3.96 | 3.5 |
index_scan | 30.26 | 53.85 | 1.8 |
oltp_point_select | 0.15 | 0.49 | 3.3 |
oltp_read_only | 2.97 | 8.58 | 2.9 |
select_random_points | 0.3 | 0.75 | 2.5 |
select_random_ranges | 0.35 | 1.14 | 3.3 |
table_scan | 30.81 | 63.32 | 2.1 |
types_table_scan | 70.55 | 189.93 | 2.7 |
reads_mean_multiplier | 2.6 |
Write Tests | MySQL | Dolt | Multiple |
---|---|---|---|
bulk_insert | 0.001 | 0.001 | 1.0 |
oltp_delete_insert | 3.02 | 13.22 | 4.4 |
oltp_insert | 1.5 | 3.13 | 2.1 |
oltp_read_write | 5.18 | 18.28 | 3.5 |
oltp_update_index | 1.55 | 6.09 | 3.9 |
oltp_update_non_index | 1.47 | 6.67 | 4.5 |
oltp_write_only | 2.35 | 9.39 | 4.0 |
types_delete_insert | 3.02 | 13.22 | 4.4 |
writes_mean_multiplier | 3.5 |
Overall Mean Multiple | 3.0 |
---|