Breaking Changes
This release externalizes the embedding model configuration. Three new required environment variables must be set before upgrading:
DENSE_EMBEDDER=(openai_text_embedding_3_large / openai_text_embedding_3_small / mistral_embed / local_minilm)
SPARSE_EMBEDDER=fastembed_bm25
EMBEDDING_DIMENSIONS=(<=3072 for openai large / <= 1536 for openai small / ==1024 for mistral / ==384 for minilm)
If you want to keep your existing data: set these variables to match the models and dimensions your instance was already using. No re-sync needed.
If you want to switch to a different model or different dimension: set the variables to your desired configuration. Note that changing the embedding model or dimensions makes existing synced data incompatible -> all collections will need to be deleted and re-synced.
What's Changed
- chore: remove connections API and remove all components by @felixschmetz in #1487
- feat: add LinearCursor and support incremental sync for LinearSource by @EwanTauran in #1486
- [code blue] OauthFlowService and OauthCallbackService + hook into API layer by @felixschmetz in #1472
- Add
codatoself-hosted.integrations.yamlby @viralpraxis in #1474 - feat: ENG-15: Implement
slabconnector by @viralpraxis in #1481 - Refactor/embedding models code blue by @marc-rutzou in #1479
Full Changelog: v0.8.99...v0.9.0