github langgenius/dify 1.12.0
v1.12.0 - Introducing Summary Index: Smarter Retrieval with AI Summarization
on GitHub

latest release: 1.12.1
16 hours ago

🚀 New Features: Summary Index

We are introducing Summary Index, a powerful enhancement to our knowledge base retrieval system that significantly improves search accuracy by generating AI-powered summaries for document chunks.

Background

Traditional vector search relies on raw chunk embeddings, which can miss semantic nuances and context when matching user queries. This is especially challenging for long documents or complex content where key information might be scattered across multiple chunks. Summary Index addresses this by creating concise, semantically-rich summaries for each document chunk, which are then vectorized and used as an additional retrieval layer.

Key Capabilities

  • AI-Powered Summarization: Automatically generates concise summaries for document chunks using configurable LLM models, capturing essential semantic information in a compact format.
  • Multimodal Support: When using vision-capable LLMs (e.g., GPT-4V, Claude-3), the system can generate summaries that incorporate both text and images from document chunks, providing richer context understanding.
  • Enhanced Retrieval Accuracy: Summary vectors serve as an additional retrieval layer, improving the precision of knowledge base searches by matching queries against both original content and AI-generated summaries.
  • Flexible Configuration: Supports default summary prompts, allowing you to tailor the summarization style to match your domain-specific requirements.
  • Asynchronous Processing: Summary generation runs asynchronously, ensuring that document indexing remains fast and non-blocking.
  • Manual Summary Editing: Allows you to manually edit and refine AI-generated summaries to better align with the original chunk content, ensuring summaries accurately reflect domain-specific terminology and context.
  • Index Structure Compatibility: Works with both general chunking and parent-child chunking modes, with intelligent handling of hierarchical document relationships.
  • High-Quality Index Integration: Available exclusively for datasets using the "high_quality" indexing technique, ensuring optimal performance for production knowledge bases.

Other Improvements

  • Agent App Multimodal Support: Agent App now natively supports multimodal inputs (images/files).
  • Qdrant Full-Text Search: Implemented full-text search with multi-keyword support for Qdrant vector database.
  • Workflow Enhancements:
    • Added support for registering and reading context in workflows.
    • Added OpenTelemetry support for single runs.
  • Plugin System: Added an option to delete or keep API keys when uninstalling a plugin.
  • Template Preview Before Creating: Added a detailed app template preview with workflow graphs or app configs by app type.

🐞 Bug Fixes & Security Updates

  • Security:
    • Logstore security fix: Addressed potential SQL injection, serialization issues, and optimized initialization.
    • Fixed an SSRF vulnerability in WordExtractor URL download.
    • Upgraded tanstack devtools to fix seroval RCE vulnerability.
    • Security updates for pdfminer.six, authlib, werkzeug, aiohttp, and pycryptodome.
  • Workflow & Graph Engine:
    • Fixed issues where single iteration or loop nodes could not run.
    • Fixed NextStep crash when the target node is missing.
    • Fixed ToolInvokeMessage Union type parsing issue.
  • API & Backend:
    • Fixed CORS resource pattern for /console/api routes.
    • Fixed IRIS hybrid search returning zero results.
    • Fixed billing account deletion issues.
    • Fixed vdb-migrate command for parent-child segment migration.
  • Frontend:
    • Fixed "Expand all chunks" button not working.
    • Fixed vertical scrolling support for floating elements.
    • Fixed validation error when uploading images with None URL values.

⚙️ Configuration Changes

  • SANDBOX_EXPIRED_RECORDS_CLEAN_TASK_LOCK_TTL: Sets the lock TTL for the sandbox expired records cleanup task (Default: 90000).
  • SMTP_LOCAL_HOSTNAME: Optional override for the local hostname used during SMTP HELO/EHLO.
  • PLUGIN_MODEL_SCHEMA_CACHE_TTL: Sets the cache TTL for plugin model schemas (Default: 3600).
  • ENABLE_TRIAL_APP: Enables trial app (Default: false).
  • ENABLE_EXPLORE_BANNER: Enables explore banner (Default: false).

What's Changed

New Contributors

Full Changelog: 1.11.3...1.12.0

Check out latest releases or
releases around langgenius/dify 1.12.0

Don't miss a new dify release

NewReleases is sending notifications on new releases.

Get notifications