github dolthub/dolt v1.8.5
1.8.5

latest releases: v1.43.0, v1.42.20, v1.42.19...
14 months ago

This is the first release that supports Full-Text Indexes.

Merged PRs

dolt

  • 6398: Implemented Full-Text indexes
    Technically functioning, although not the most ergonomic implementation. I need add a bit of user convenience and additional checking, such as removing the additional tables from dolt status, automatically adding the additional tables when the parent is added via dolt add, adding a user-facing write barrier on the tables, etc. Also need quite a few more tests, but the basics are in via the engine tests and very, very few bats tests.
    I'm definitely going to expand the testing in the future, as well as implementing the aforementioned features and more. But for now, this should at least allow us to release unless there are major problems that I've not caught.

go-mysql-server

  • 1871: Full-Text Indexes
    This is a partial implementation of Full-Text indexes. There is still quite a bit to finish on the GMS side (as can be seen from the copious amount of TODOs), but this shows the broad strokes of how it's implemented, along with most of the "difficult" design choices being implemented. The major choice that has not yet been finalized is how to deal with FTS_DOC_ID, as it's an AUTO_INCREMENT column in MySQL, but that would not play well with Dolt merging. I already have ideas on how to handle that (taking into account remotes, etc.), but that would come from a later PR.
    https://docs.google.com/document/d/1nGyYg461AhxQjFLzhEEj01XMz0VaTBaBaA44WNu0fc4/edit
    Quite a few things have changed from the initial design doc, mostly based on feedback during the meeting, however some of it was post-meeting. There are three tables instead of 1: Config (stores table-specific information shared across all indexes), WordToPos (maps words to an ID and position, not fully used in the default search), and Count (used to calculate relevancy, also not fully used in the default search). I was planning on converting MATCH ... AGAINST ... to a join between the tables, which would work when fetching results, but MATCH ... AGAINST ... may also be used as a result, which necessitated writing all of the functionality anyway, so the join plan was dropped.
    Last thing to mention, is that I'm pretty sure that Full-Text indexes actually do a full table scan. It seems weird, but AFAICT the indexes are used to quickly calculate relevancy for each search mode. It seems that, for overly large tables, the search time increases even when other index operations continue to operate nearly instantaneously.
    I've tagged two people for review to make it a bit easier. Of course, feel free to take a look at more if you desire.
    @reltuk The sql/fulltext/fulltext.go file is an expansion of the file you've previously reviewed (all still kept to a single file for now). To complement it and see how it'll be implemented on the Dolt side, you can look at memory/table.go. Dolt's table editor will be similar, and the merge paths will only use the FulltextEditor, which special logic to interface with it from those paths.
    @max-hoffman Take a look at the analyzer changes, along with the sql/plan/ddl.go file. You'll probably need to reference sql/fulltext/fulltext.go as well.
  • 1863: Bump google.golang.org/grpc from 1.37.0 to 1.53.0
    Bumps google.golang.org/grpc from 1.37.0 to 1.53.0.

Closed Issues

  • 6382: mysql_server may panic when performing a join query on dolt_history_$tablename and dolt_commits tables.
  • 1902: Valid join query erroneously reported as ambiguous
  • 1881: duplicate column names in a table alias should be prevented.
  • 1753: "SELECT * FROM tbl" has incorrect schema when table has default values.
  • 1745: Allow table names with special characters when MySql allows them

Latency

Read Tests MySQL Dolt Multiple
covering_index_scan 2.07 3.02 1.5
groupby_scan 13.22 17.95 1.4
index_join 1.25 4.74 3.8
index_join_scan 1.21 2.3 1.9
index_scan 32.53 57.87 1.8
oltp_point_select 0.14 0.48 3.4
oltp_read_only 2.66 8.28 3.1
select_random_points 0.3 0.81 2.7
select_random_ranges 0.37 1.18 3.2
table_scan 32.53 56.84 1.7
types_table_scan 74.46 167.44 2.2
reads_mean_multiplier 2.4
Write Tests MySQL Dolt Multiple
bulk_insert 0.001 0.001 1.0
oltp_delete_insert 4.74 5.88 1.2
oltp_insert 2.22 2.97 1.3
oltp_read_write 5.88 15.83 2.7
oltp_update_index 2.3 3.02 1.3
oltp_update_non_index 2.3 3.02 1.3
oltp_write_only 3.36 7.56 2.2
types_delete_insert 4.91 6.55 1.3
writes_mean_multiplier 1.5
Overall Mean Multiple 2.1

Don't miss a new dolt release

NewReleases is sending notifications on new releases.