Merged PRs

dolt

• 2934: integration-tests/transactions: Added Concurrent Transaction Tests
• 2933: go/utils/remotesrv: use io.Copy in the http file server to write table file data to http.ResponseWriter

  Problem

  This PR fixes an issue when pulling chunks from Dolt remotesrv. The symptom is a throughput below minimum allowable error. The problem is that the dolt client can't handle large http chunks without failing to meet its minimum required download throughput.
  Dolt remotesrv hosts a simple http file server that can be used to write new table files and read existing table files. When remotesrv receives a request to read a file it will currently read all of the requested range into memory. It then, writes the
  entire range to http.ResponseWriter in a single Write call.
  This causes a very large http chunk (see chunked transfer encoding) to be sent over the wire. When the Dolt client tries to read the http response using Read, it takes a long time to return. Due to the way the Dolt client measures Read throughput, if a single Read takes long enough, the minimum required throughput will not be met.

  Solution

  We fixed this issue by using io.Copy to read from the file and simultaneously write to the http.ResponseWriter. io.Copy produces small enough writes to the response writer. This reduces the number of bytes read during a singular Read call on the Dolt client and allows the throughput measurement to be accurate.
  We could have fixed this problem by changing the way the minimum required throughput was measured, but the above change reduces the memory overhead of remotesrv and uses an idiomatic Golang pattern.

  Why doesn't this issue appear when pulling chunks from S3?

  In Golang, http.Response.Body maximum Read sizes are determined by the Content-Length and whether chunked transfer encoding is being used. The exact semantics can be investigated by reading the sources here:
  • net/http parseTransferEncoding
  • net/http chunkedReader
  • net/http chunkedWriter
    Since S3 returns a Content-Length and does not use chunked transfer encoding each Read call returns in a small enough size.
• 2925: add parquet to dump options
• 2923: Column union works like a set union
• 2912: SQL dump now has --batch option for batched INSERTS
• 2901: Edit Stats display in Import and fix stats computation
  This pr
  1. Fixes the formatting of stats and allows commas to be printed
  2. Fixes an error with stats computation where many queries were getting marked as modifications.
  3. Bumps GMS to add in a fix with dropped insert ignore errors
• 2900: Added skipped bats for dolt version requiring write perms
• 2895: Allow Rollback on Keyless Tables
  Delete all pending changes when an error occurs.

go-mysql-server

• 851: add collation_character_set_applicability table in infoSchema
• 850: Added sql.FilteredIndex to allow integrators to specify index filter pushdowns
• 846: analyzer: Collect all range expressions for multi-column indexes
• 845: Fix InTuple optimizer bug
  Last week we fixed a dropped optimizer error return that was masking a
  tuple bug. We expected a slice of interfaces, but single element tuples
  return one value. We catch this in most circumstances, but missed the
  multicolumn case triggered by an AND expression. The error was passing
  logictests because we still return correct results even if we fail to
  push a filter into a table scan.
• 842: /.github/workflows/bump-dependency.yaml: pin go version
• 841: Bump vitess while automatic bumping is broken
• 840: /.github/workflows/bump-dependency.yaml: add GOOS env
• 839: server: Refactor handler to use buffered channel
• 837: Add IGNORE behavior for CHECK CONSTRAINTS and on duplicate key
  Fixes an issue where INSERT IGNORE was not correctly ignoring check constraints
• 836: fix set serialization
• 833: Fixing a race condition by no longer updating the UserVar's type during Eval
  The type was being set when retrieved in Eval for completeness, but it shouldn't be needed.
  When used as a value, the analyzer was already using the NewUserVarWithType method that explicitly loaded and set the type. The NewUserVar method (which doesn't set type) should only be used when the user var is the left hand side of a set var operation.
• 832: add processlist table to information_schema
  The result is the same as current SHOW PROCESSLIST query result
• 829: Added additional privilege tests
  Added new testing structure for privileges I'm calling "quick tests". A test can take up 6 lines rather than 18, which is a big difference when there's 100 of them. Plus they're quicker to write this way. They're the same is normal tests though.
• 825: add information_schema.statistics table
  Cardinality is currently set to total number of row in the table instead of an estimate number of unique values in the index

vitess

• 132: Add FLUSH as a command and its options
• 130: add PROCESSLIST to non-reserved-keyword list
• 129: Adding scripts to test parsing behavior of MySQL
• 128: /go/bucketpool: Optimized findBucket
• 127: Add testcases for exercising keywords and remove some create user tests
  This PR:
  1. Allows the COMMENT_KEYWORD to be alias by selects/
  2. Removes some usages of aliased keywords in the select queries
  3. Adds skipped tests that exercises the validity of reserved and non reserved keywords through a series of tests
• 126: go/mysql: Add buffering to prepared statement result sets

Closed Issues

• 2865: dolt dump and dolt diff --sql should feature a --batch flag
• 2919: 3-way schema merge creates duplicate columns
• 1843: Bad error message when database is unselected
• 2615: dolt table import -u <table> shows incorrect processing information for us-housing-prices
• 2630: Support flag that skips over printing skipped rows in import
• 2814: Syntax error when use 'Comment' as column alias
• 2310: [Feature Request/Bug] Add processlist to information_schema table
• 2856: Add details to information_schema.statistics table.
• 2890: Keyless Tables Do Not Support Rollback Semantics
• 2897: can not exit after execute dolt login
• 2692: SHOW CREATE TABLE does not respect primary key ordinals