We are excited to announce the release of Dolt 0.17.1.
In this lease we extend the index support we discussed in the prior release to UNIQUE
indexes. Our type inference now also supports DATE
, TIME
, and DATETIME
types.
Merged PRs
- 713: removed pkg resultset
- 710: New TableEditor, thread-safety, and table import
So I created a newTableEditor
underdoltdb
, similar to theIndexEditor
. I renamed the old editor tosqlTableEditor
, and it's now usingTableEditor
underneath. It's just to comply with the SQL interfaces.
Perhaps one of the most striking changes are the synchronization changes. The motivation was from a comment from @zachmu , which implied that some of the write paths are parallelized for performance, and the old editor was not thread-safe by any means. In order to support these new write paths, it was necessary to makeTableEditor
thread-safe. Because of this,IndexEditor
also received the same changes. I verified that there was a lack of data races usinggo test -race
, and also ran the concurrency tests with thousands more iterations and edits (they're lower now so that we're not spending an hour in Jenkins). Just to verify that my tests also worked, I made temporary intentional changes that would allow for data races, and the tests caught them, andgo test -race
also caught them.
In addition to the above changes,dolt table import
anddolt table cp
both use theTableEditor
now. Due to the concerns mentioned in the previous paragraph, I decided to benchmark a few metrics to get an idea of the performance impact of these two major changes.
To measure just the impact of the threading changes, I imported 1,000,000 rows through SQL and timed the results:
The result is close enough that it's within the margin of error, so it's safe to say that newBefore: 67.074s After: 67.363s
TableEditor
is just as performant as the old implementation.
Thetable import
is a different story though. This time, I imported a 10,000,000 row.psv
, with the results being:
It takes roughly twice as long now, which is slower, but not tragically so. Primarily, the difference in speed is that the old code just wrote directly to the edit accumulator, where as theBefore: 53.521s After: 107.816s
TableEditor
does a ton more bookkeeping, being a generalized editor. - 709: testing large numeric types
- 708: Extra BATS for
table import
&schema import
- 707: validates index columns
- 706: reset hard fixes
- 705: always calculate merge stats
- 703: Zachmu/indexes
Killed off index driver and let tables declare indexes natively. - 702: create table bats
- 701: Bh/remotes bat fixes
- 700: swish
- 699: Andy/arg parsing
- 698: fixed some bats tests
- 697: support HEAD and ancestor spec syntax
- 694: fixing zach's bats
- 693: optimize diff and hist tables
- 692: floats literals can be used in int columns in mysql 8
- 691: quoting time types
- 690: fix panic for
select 1/0 from dual
depends on go-mysql-server PR - 689: fixed panic for empty string args
one down - 688: o/libraries/doltcore/remotestorage/events_interceptor.go: Add missing ADD_TABLE_FILES instrumentation.
- 687: go/{store/nbs,libraries/doltcore/remotestorage}: Expose repository size as Size() on TableFileSource.
- 686: go/libraries/doltcore/remotestorage: Move retrying and metrics instrumentation for doltremoteapi interactions to grpc client interceptors.
- 685: go/store/nbs/table.go: Thread ctx at Open() for table file pull interface.
- 684: proto: remoatesapi: GetRepoMetadataResponse: Add repository_size field for communicating approximate repository size.
- 683: Added UNIQUE constraint
Largest change you'll see here is that I've extracted the index changes intable_editor.go
into its own file and built upon it. I've also changed everything that modifies indexes, from rebuilding the indexes to the SQL path, to make use of the new index editor. - 682: table import inferrence
- 681: Diff SQL handles escape sequences
- 679: Index performance improvements
- 678: diff v2
- 677: Updated version for release of version 0.17.0
- 110: Zachmu/index refactor
Rewrote index interfaces. Indexes are now available either through a driver, or natively via the table implementations themselves.
Also re-enabled auth tests for windows by removing the pilosa dependency, and fixed several test suites that were broken on Linux. - 109: 1.0/0.0 == NULL, 1 div 0 == NULL
- 108: Andy/div by zero
This fixes a panic forselect 1/0 from dual
MySQL also errors on divide by zero in at least some configs
I can change this to returnNULL
if that's preferable. - 107: sql/session.go: Use new view and index registries per sql.Context, instead of global ones.
- 106: Zachmu/table naming regression
Fixed regression caused by update to alias handling. Removed a couple places that allowed aliased tables to be referred to by their unaliased names, which is an error.