weaviate/weaviate v1.22.0

v1.22.0 - Async Indexing, Nested Object Support, gRPC API Support, Schema Repair, OIDC Group Auth, and many other improvements & fixes

on GitHub

Breaking Changes

none

New Features

Async Indexing

Support for asynchronous indexing is now available! This feature is experimental, and opt-in. To use it, set the environment variable ASYNC_INDEXING=true. With this, Weaviate now offers you the option de-coupling vector indexing from object creation. You can enable this to optimize imports for speed where vector indexing is not required immediately.

• Share workers between queues by @asdine in #3590
• Index stale queue items by @asdine in #3591
• Add new shard INDEXING state by @asdine in #3596
• Graceful shutdown by @asdine in #3625
• Skip indexed vectors at startup by @asdine in #3568
• Fix brute force search with cosine distance by @asdine in #3636
• Improve import time using leaky bucket strategy by @asdine in #3642
• Add async acceptance test workflow by @asdine in #3652
• Update REST API to allow for broadcasting of shard statuses by @tsmith023 in #3673
• Add vectorQueueSize to /schema/{className}/shards response by @tsmith023 in #3682
• Introduce async indexing by @asdine in #3424

Nested Object Support

You asked for it, now you have it! Store full objects directly into Weaviate. Have unexpected or dynamic object properties? No problem, nested object support is supported by autoschema as well, meaning the schema will be updated dynamically according to object properties.

• Add support for object/object[] data types by @aliszka in #3600
• Autoschema improvements by @aliszka in #3626
• Run autoschema on object's update/merge allowing creating new props or nested props by @aliszka in #3641

gRPC API Support

Weaviate now officially supports gRPC, significantly improving transport efficiency! The easiest way to take advantage of this feature is to use one of the official language clients. Please note that if you were using gRPC beforehand, you will need to upgrade to use the latest gRPC services.

• Implement gRPC health check by @antas-marcin in #3587
• Split protofiles into v0 and v1 packages for clear API break by @tsmith023 in #3639
• Implement gRPC Nested Objects by @tsmith023 in #3601
• Adjust gRPC Java package name by @antas-marcin in #3665

Schema Repair

• Repair schema when cluster is out of sync by @parkerduckworth in #3575

Module Enhancements

• Add gpt-3.5-turbo-instruct to available qna-openai models by @xysun in #3574
• Add support for vectorizing text[] props in multi2vec-bind by @tsmith023 in #3654
• Add support for vectorizing text[] props in multi2vec-clip by @tsmith023 in #3653

Performance Optimizations

• Improve Startup Time by initializing Shards in parallel by @etiennedi in #3658
• Improve Shutdown speed by shutting down shards in parallel by @etiennedi in #3661
• Removes tombstones when merging with root segment by @aliszka in #3666
• Add memory guard rails for batch creation by @etiennedi in #3582

OIDC Group Auth

• OIDC group authentication support by @doublefelix7 in #3581

Other

• Refactor core object to allow scripting by @donomii in #3612
• Add help command to test run script by @reyreaud-l in #3628
• Modify ContainsAny / All tests to run also in cluster setup by @antas-marcin in #3678
• Backup restore add node_mapping parameter by @reyreaud-l in #3602

Fixes

• Refactor beacon parsing in reference target autodetection by @dirkkul in #3420
• Fix flaky test of cycle callbacks by @aliszka in #3404
• Disallow changes to distance setting after creating a class by @nilskulawiak in #3376
• Remove superfluous backup read locks by @redouan-rhazouani in #3492
• Bump dev prometheus, add host metrics dashboard by @trengrj in #3491
• Adds ContainsAny/All tests for text properties by @aliszka in #3498
• Add needs-investigation to don't-close list by @etiennedi in #3511
• Fix panic with negative value of k by @amityahav in #3512
• Enable unused, nolintlint; fix lint issues by @alexandear in #3531
• Fix typos in comments and tests by @alexandear in #3530
• Fix: unhandled batch delete error by @parkerduckworth in #3549
• Hnsw: turn neurips23 test into benchmark by @asdine in #3393
• Set missing name attribute in workflow files by @aduis in #3578
• Make proto naming consistent by @dirkkul in #3576
• Update docker compose doc link by @cedarkuo in #3330
• Fix vectorization of concepts in nearText argument by @antas-marcin in #3494
• Fix gRPC Batch API AutoSchema by @antas-marcin in #3657
• Fix for GraphQL ref search between MT and non-MT classes by @aliszka in #3679

New Contributors

• @amityahav made their first contribution in #3512
• @aduis made their first contribution in #3578
• @xysun made their first contribution in #3574
• @cedarkuo made their first contribution in #3330
• @doublefelix7 made their first contribution in #3581

Full Changelog: v1.21.7...v1.22.0