vaticle/typedb 2.1.0

TypeDB 2.1.0

on GitHub

Install & Run: http://docs.vaticle.com/docs/running-typedb/install-and-run

New Features

• Concludable and Retrievable Subsumption Caching
  We introduce caching in the reasoner avoid repeated work. Adding this caching has also allowed us to tighten the criteria upon which we trigger an outer reiteration of the reasoner for a given query. Such an iteration is necessary to search for inferred facts that can only be found based on the inferences made by prior iterations.
  The cache ensures that work can be re-used for the same exact partial answer when revisited. We also implement cache subsumption: any partial answer can utilise the cached answers for any more general (subsuming) partial answer which has a complete cache.

• Restore ACID transactionality with Consistency Manager
  We restore TypeDB's ACID transactionality by fixing the two existing holes with Consistency: keys were able to be duplicated in concurrent transactions, and concurrent deletion and insertion could lead to corrupted data. This PR validates conflicting keys using a new WriteConsistencyManager.

Bugs Fixed

• Request reiteration status with explicit resolver messages
  Within the message-passing framework of the reasoner, answers would carry a flag to indicate whether an additional outer iteration is necessary to find all answers. It was possible that the answers needed to carry this flag could be not present or swallowed, and so the answer produced would not reiterate. In recursive cases in particular this caused missed answers. We solve this by introducing request and response messages specifically to communicate whether reiteration is required.

• Statistics background thread catches correct conflict codes
  As of #6306 we have enabled ACID transactionality, but the background statistics counter did not catch the correct error codes that represent conflict operations and simply died. We now catch the correct error codes and ensure statistics are still updated in the background thread.

• Reduce OOM by reusing rocks iterators in write transactions
  To fix issues like #6323, we identify that the cause is that rocks iterators are not re-used in write transactions, and also not immediately cleaned up. We now allow write transactions to re-use rocks iterators as it they do in fact see uncommitted deletes to the rocks transaction.

• Fix parallelised limited disjunctions
  Running a disjunction with parallelism enabled, and a limit should terminate with the right number of answers. However, it can sometimes stall completely, which is addressed by correctly resetting the number of answers being processed when one disjunction finds no answers.

• Rule redefinition deletes old rule completely
  Undefining or overwriting a rule incompletely deletes the previous rule before writing the new rule. This leads to #6311 and #6309, and is fixed by deleting a Rule completely.

• Fix traversal backtrack seek and don't ignore types given IID
  This PR fixes two traversal bugs: one backtracking bug that leads to #6303 and a bug found by @jmsfltchr when using IIDs with types which led to types being ignored. We also remove IID lookups from TypeResolver, as it should not look at data.

• Fix validation for adding @key to owned attribute type
  When migrating owns to owns @key in the schema, we were incorrectly checking every attribute is owned once -- we should check that every attribute that is owned, is only owned once. We now correctly validate that

  1. all instances that should have a key have exactly 1 key attribute
  2. every attribute keying an instance of a type is only keying exactly 1 instance of that type

Code Refactors

• Reintegrate reverted unification tests
  We re-update #6114 , which was merged and reverted due to an error in master.

• Implement viewable byte array wrapper class
  We implement a custom byte array wrapper to expose hash and equality functions, as well as implementing the Comparable interface. We also implement ByteArray.View to reduce new array allocations, which appears to improve performance of bulk writes by about 5% (probably due to fewer GC operations), but has no overall impact on mixed read/write performance.

• Clarify error thrown during set abstract
  We throw an unhelpful error when setting a type as abstract stating the type cannot be deleted. We clarify the error message to be specific about setting abstractness.

Other Improvements