langgenius/dify 0.3.31

v0.3.31

on GitHub

New Features

• Added support for multi-datasets retrieval, hybrid search, and result re-ranking.
  Learn more: Dify.AI v0.3.31: Surpassing the Assistants API – Dify's RAG Demonstrates an Impressive 20% Improvement

Important Upgrade Notice

Current version adds full-text search and hybrid search functions for datasets, significantly improving search results. If you want to use this feature, you need to make the following configurations:

• If the vector database you are using is Weaviate
  Please update your version to v1.19.0 or later in /dify/docker/docker-compose.yaml.
  For more details, please refer to the Weaviate release list.

 weaviate:
    image: semitechnologies/weaviate:1.19.0
    restart: always
    volumes:
      # Mount the Weaviate data directory to the container.
      - ./volumes/weaviate:/var/lib/weaviate
    environment:
      # The Weaviate configurations
      # You can refer to the [Weaviate](https://weaviate.io/developers/weaviate/config-refs/env-vars) documentation for more information.
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'false'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'none'
      CLUSTER_HOSTNAME: 'node1'
      AUTHENTICATION_APIKEY_ENABLED: 'true'
      AUTHENTICATION_APIKEY_ALLOWED_KEYS: 'WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih'
      AUTHENTICATION_APIKEY_USERS: 'hello@dify.ai'
      AUTHORIZATION_ADMINLIST_ENABLED: 'true'
      AUTHORIZATION_ADMINLIST_USERS: 'hello@dify.ai'

• If the vector database you are using is Qdrant
  1. Since Dify's full-text index relies on the multilingual tokenizer in Qdrant's full-text index, by default, Qdrant does not support Chinese, Japanese, and Korean language tokenization.
  2. Therefore, we provide a full-language tokenizer mirror for Qdrant: langgenius/qdrant:latest, which can be deployed and used directly. You can also build your own Qdrant image using the following command:

  docker buildx build .  --build-arg  FEATURES="multiling-chinese,multiling-japanese,multiling-korean" --tag=langgenius/qdrant
  

  1. Due to the lack of full-text index in previous Qdrant data, you need to execute the following commands in the api container to complete the process:
     1. Start a bash shell inside the running Docker container:

      docker exec -it docker-api-1 bash
     

     1. Execute the command to create the full-text-index index:

       flask add-qdrant-full-text-index
     

• Milvus/Zilliz temporarily does not support Hybrid search

What's Changed

• Update README.md by @goocarlos in #1525
• fix: conversation rename always auto generate by @takatost in #1530
• new readme by @crazywoola in #1528
• doc: update readme by @crazywoola in #1536
• feat: optimize timezone of server by @takatost in #1537
• Doc/update readme patch 1 by @crazywoola in #1538
• feat: add code of conduct by @crazywoola in #1541
• fix: upload image by @zxhlyh in #1522
• feat: remove plugin page by @zxhlyh in #1544
• update images by @crazywoola in #1548
• update images by @crazywoola in #1549
• fix: upload file not clickable in firefox by @crazywoola in #1552
• feat: supports for new version of openllm by @takatost in #1554
• Add some interesting badges :) by @crazywoola in #1558
• Feat/add retriever rerank by @JohnJyong in #1560
• Feat/jp and es by @crazywoola in #1562
• feat: add retriever rank fe by @iamjoel in #1557
• fix hybrid search reranking check by @JohnJyong in #1563
• fix the error message by @JohnJyong in #1564
• chore: retrieval docs links and enchance help doc translation by @iamjoel in #1570
• feat: fetch stream compatibility enhance by @rickythink in #1551
• fix(web): Sidebar create new chat context by @JunIce in #1569
• Feat/sdk vision support by @GarfieldDai in #1531
• fix(api): patch Windows timezone set by @JunIce in #1575
• feat: configurable invite expiry time by @MatriQ in #1573
• Fix/check rerank model selected by @zxhlyh in #1577
• fix: some text by @zxhlyh in #1579
• fix multi retrieval with resource score issue by @JohnJyong in #1578
• feat: add supported_model_types field and filter in provider list by @takatost in #1581
• Compatible model saving error by @JohnJyong in #1582
• feat: bump version to 0.3.31 by @takatost in #1584

New Contributors

• @rickythink made their first contribution in #1551

Full Changelog: 0.3.30...0.3.31