vaticle/typedb 1.7.0

Grakn Core 1.7.0

on GitHub

Install & Run: http://dev.grakn.ai/docs/running-grakn/install-and-run

New Features

• Include rule in explanation.
  When returning an explanation, we now return the rule from which the explanation is derived.
  Requires:
  vaticle/typedb-protocol#24
  vaticle/typedb-driver#81

• Add stricter rule validation for unplayable roles.
  Validate rules more strictly, banning all occurences of rules that infer a role that can never be played for any possible type that the role player variable may be.

Bugs Fixed

• Defer db completeness check in query cache when fetching.
  When fetching answers from the query cache, we are currently checking for query completeness at the very beginning of the call. What is possible however is that we propagate answers from a complete parent and hence acknowledge a child complete. As a result, currently we would still resort to doing a DB lookup where all answers are actually available in the cache. Here, we correct this behaviour by deferring the completeness check to after potential answer propagation.

• Validate resolution profiler answers.
  Resolve issues with incomplete/out of sync answers in the resolution profiler.

• Make resolution query profiler output directional.
  We make the resolution query profile explicitly directional (edges have a strict direction). It helps removing ambiguity when reading.

• Add query profiler based on resolution tree.
  To add a better way of examining query execution with reasoning by providing a way of outputting the resolution tree in a graphical-friendly form. The tree contains both answer and processing time per node information.

• Fix the equality comparison of AttributeAtoms with differing attribute variables 2.
  We add a fix for the comparison of AttributeAtoms with different variables, but the same owner variable, and the same attribute type.

• Fix and Implement Direct Sub.
  #5628 flags that direct sub is broken, and #4473 indicates reasoning is not able to handle direct Sub at all. This PR implements direct Sub in reasoner, and removes the reasoning code from the non-reasoning code path to fix the direct sub when reasoning is disabled.

• Do not parse patterns when converting attributes to isas.
  To avoid parsing patterns when doing AttributeAtom->IsaAtom conversion as it is expensive. Instead, we try to use atoms at hand.

• Remove excess unification when comparing RelationAtoms.
  We remove an accidentally introduced extra unification step when comparing RelationAtoms introduced in one of the previous PRs.

• Store labels instead of ids in atoms.
  To store SchemaConcept labels instead of their ids in atoms.
  Reasons:

  1. ids carry little information and require further manipulation (lookups) to extract their meaning
  2. we often want to use the label which then requires doing a lookup via ConceptManager.
     As we trim down the number of dependencies on ConceptManager, this change also should allow to remove ReasoningContext out of atoms and instead inject it to functions that require it.

• Also log the type of a Thing that needs a key.
  We have improved the error message that is displayed when you're not inserting the key of a concept that needs it. Not knowing the type of the concept you are trying to insert as you try to fix the query is quite painful.
  The error message before this PR:

  Thing [V4320] does not have exactly one key of type [symbol]. It either has no keys assigned to it, or it has more than one. 
  

  The error message after this PR:

  Thing [V4320] of type [commit] does not have exactly one key of type [symbol]. It either has no keys assigned to it, or it has more than one. 
  

Code Refactors

• Abstract atom conversions.
  To delegate the atom conversion task in Atoms to a new AtomConverter entity so that the Atom implementations are simpler so and we can eventually make them sole data classes.

• Removed unused dependencies in Java classes.
  We would like to clean-up all unused dependencies in Java class import statements, so that we can start refactoring the codebase to find package boundaries.

• Abstract type reasoning in atoms.
  To delegate the type inference tasks in Atoms to a new TypeReasoner entity so that the Atom implementations hold less responsibility and are less crowded - easier to maintain and understand.

• Remove operator package.
  We remove the operator package from the repo as it is going to be used in multiple different projects and henceforth deserves a separate repo.

• Simplification of Grakn MapReduce Framework.
  Our MapReduce framework, which previously sits in a package called //graph-hadoop (with package address (grakn.core.graph.hadoop), was inherited from JanusGraph and has a very complex implementation. This PR attempts to simplify the codebase by simplifying the high-level structure of the class hierarchy and dependency. The package is now called //mapreduce.

• Extract TraversalExecutor from QueryExecutor.
  Refactor QueryExecutorImpl, which is instantiated per-query with an infer flag. Interally it used to perform traversals on the backend, which was completely independent of inference. This PR extracts TraversalExecutor out of QueryExecutor, which has the side effect of removing ExecutorFactory from much of reasoner and only passing a TraversalExecutor directly. This makes more sense than before, as the semantics of holding a ExecutorFactory indicates reasoner might perform read, write, or compute queries where it really only needs to perform read traversals using the TraversalFactory.

• remove all usage and references to old benchmark.
  Remove old references to old benchmarking code, including tracing and benchmark flag.

• Merge Query Execution Code Paths for Reasoning and Non-reasoning.
  This PR merges the entry point for reasoning and non-reasoning queries to use the type inference code, but then diverges in the resolve() method within ReasonerQueryImpl. This means we push the infer=true/false flag further down the stack, using a ResolutionIterator to resolve with inferece versus traverse(pattern) to just read the DB.

• Remove BasicSemanticProcessor.
  Remove the BasicSemanticProcessor so that the unification flow is clearer.

• Cleanup atomic hashing.
  To simplify and clarify our atom hashing functions.

• Flatten RelationAtom.
  Instead of extending IsaAtom, we make the RelationAtom contain the associated IsaAtom. As a result the Atom hierarchy is more flat and simple and we enable further simplifications.

• Merge Binary and TypeAtom.
  We merge Binary and TypeAtom as the differentiation is needless and they share exactly the same children.

• Flatten AttributeAtom.
  Instead of extending Binary, we make the AttributeAtom contain relevant IsaAtoms so that the Atom hierarchy is simpler and we enable further simplifications.

• Atom cleanup.
  Cleanup the code after introducing the different task abstractions in previous PRs.

Other Improvements

• Fix release validation for @graknlabs_console.
  Allow us to proceed with release

• Fix the path to bash.exe in test-assembly-windows-zip.
  We have fixed the path to bash.exe to point to the correct path.

• CI fixes.
  Previously, CI failed because of changes caused by Bazel upgrade. Additionally, Bazel on Windows also needs to be bumped to the latest version

• Fix excessive rebuilding.
  Previously, when switching between IDE and terminal builds, build cache would be fully invalidated. due to changed $PATH which defeats the sole purpose of caching. As per suggestion in this thread, we're enabling strict environment for actions (which sets PATH alongside with other variables to minimally viable preconfigured values).

• Update GenerativeOperationalIT to use the new fuzzying operator.
  To update the graknlabs/verification dependency to use the updated fuzzying operator.

• Add extra equivalence tests to GenerativeOperationIT.
  To add more equivalence tests to our reasoner operational test suite.

• Fix nondeterministic query resolution by extending requiresReiteration.
  Issue #5502 highlights a flaky test in reasoning that throws when a particular order of query resolution is picked. This is fixed by extending the reiteration criterion, which re-performs the resolution to find answers that might have been missed the first time around.

• Add grabl tracing.
  To add grabl-tracing to grakn server. The tracing continues from ditributed trace data sent over the protocol's metadata.

• Checkstyle: update copyright header to 2020.
  In order for build not to break after vaticle/dependencies#122, copyright headers need to be updated.

• Make generative tests exhaustive for the binary starting relation.
  Increase the set of test pairs in GenerativeOperationIT so that we can test between (child, parent) pairs across the whole generated pattern tree.

• Add basic reasoner operational test for attributes.
  To use the newly defined GeneraliseAttributeOperator to test unification facilities in the context of attribute atoms. We want to extend the GenerativeOperationalIT to cover basic cases with attributes.

• Add alpha-equivalence generative test.
  We use the VariableFuzzyingOperator to randomise variables of patterns created for GenerativeOperationalIT so that we can test exact unification and alpha-equivalence.

• Introduce generated reasoner operational tests.
  Using our subsumption operator tool, we generate large number of query pattern pairs (currently a subset of all possibilities) such that each pair is in a subsumption relationship (one pattern generifies the other). Knowing that this property holds between the pairs, we can then test our query subsumption, query unification and query equivalence algorithms as their output is known based on the subsumption relationship.