✨ What’s New in v0.6.12? ✨
Important
In the current version:
- We have deprecated
pip
as the primary package management tool in favor ofPoetry
for the Dify API service. - We have optimized the self-hosted deployment experience with Docker Compose by adding support for
.env
files and allowing the option to independently select Vector Database at startup.
The previous Docker Compose configuration has been moved to thedocker-legacy
directory.
For more details seeUpgrade to new docker compose deployment
below.
Hey everyone, we’ve got a fresh update for you with v0.6.12, and it’s packed with new features, tools, and improvements. Let’s dive in:
🚀 New Features
- Tracing with LangSmith and Langfuse: Now you can trace your apps traffic with LangSmith and Langfuse, thanks to @ZhouhaoJiang in #5483 and @iamjoel in #5487.
-
Undo/Redo for Workflow Editor: Finally, you can undo and redo your actions in the workflow editor by @perzeuss in #3927.
-
Import and Overwrite Workflow DSL: Importing and overwriting workflow DSL is now supported, you can now manage versions indirectly through DSL Export & Import by @takatost in #5511.
-
New Icons: We’ve replaced the old icons with fresh new ones by @zxhlyh in #5412.
-
Hide Workflow Steps: You now have the option to hide workflow steps in WebApp settings by @crazywoola in #5436.
-
Self-Hosting for Firecrawl Tool: The Firecrawl tool now supports self-hosting by @keita69 in #5528.
-
Latex in Chat Messages: Support for displaying Latex in chat messages by @ZuzooVn in #5001.
-
Citations and Attributions: These are now enabled by default by @takatost in #5508.
🛠️ Tools
- JSON Process Tool: Added a new tool for processing JSON by @scotlandowl in #5555.
- Serply Search Tools: Added Serply Web/Job/Scholar/News Search tools for more options by @googio in #5186.
- Novita AI Image Generation Tool: This tool now supports model search, text-to-image, and create tile functionalities by @XiaoLey in #5308.
- Create JSON Message API in Tool: Added an implements for creating JSON messages in Tool response by @hjlarry in #5440.
🤖 Model Support
-
Jina New Pre-defined Rerankers: Including jina-reranker-v2 by @takatost in #5657.
-
Bedrock Command R Models: Added support for these models by @realjustinwu in #4521.
-
Claude-3-5-Sonnet-20240620: Support added for this model by @takatost in #5452 and @soulteary in #5461.
-
Vertex AI Claude-3-5-Sonnet@20240620: Support added by @hellof20 in #5475.
-
Predefined Models for OpenRouter: Added support by @sinomoe in #5494.
-
Spark 4.0: Added support by @leslie2046 in #5688.
🗄️ Vector Database & Storage
- OpenSearch as a Vector Datasource: Support added by @baojingyu in #5322.
- Oracle23ai as a Vector Datasource: Added support by @tmuife in #5342.
- Tencent COS Storage: Support added by @quicksandznzn in #5297.
🏗️ API Service Infrastructures
- Improved Deployment Flow: Added .env to Docker Compose for better deployment by @guchenhe in #4299.
- Pydantic Settings for Config: Introduced for config definition and validation by @bowenliang123 in #5202.
- Poetry for Dependencies: Now supported in the API’s Dockerfile by @bowenliang123 in #5105.
- Removed Pip Support: Pip support for the API service has been removed by @takatost in #5453.
For additional enhancements and updates, refer to the What's Changed section below.
🎉 Upgrade to new docker compose deployment
Key Changes
-
Persistent Custom Environment Variables: Users can now declare environment variables in .env that remain persistent across deployments. This change ensures that custom configurations are not lost during upgrades or redeployments.
-
Unified Vector Database Services: All vector database services have been consolidated into a single Docker file. Users can switch between different vector databases by modifying the
VECTOR_STORE
environment variable in your local.env
. This simplifies the management of various vdb services and allows for more dynamic deployments. -
Mandatory .env File: Unlike the previous deployment method, the new method requires an [.env] file before running
docker compose up
. Users should start by copying the.env.example
file to create their own.env
file. This step is crucial for configuring the deployment to meet specific needs. -
Legacy Support: The previous deployment files have been moved to a
docker-legacy
directory. These files will no longer be maintained, but they are available for use if needed.
Migrating
For existing users who's customized docker-compose.yaml
, ssrf_proxy/squid.conf
or nginx/conf.d/default.conf
, you may need to modify the corresponding environment variables in .env
to reflect your existing changes.
We are actively seeking feedback from the community to refine and enhance the deployment process. As more users adopt this new method, we will continue to make improvements based on your experiences and suggestions.
How to Get Started
To get started with the new deployment method:
- Ensure you have Docker and Docker Compose installed on your system.
- Navigate to the new
docker
directory. - Copy the
docker/.env.example
file to a new file named.env
and customize it as needed. - Run
docker compose up
to start the services.
We encourage all users to transition to this new method to take advantage of the improved flexibility and configuration options. Your feedback during this beta phase is invaluable and will help us make the deployment process even better.
For detailed configuration options and environment variable settings, refer to the .env.example
file and the Docker Compose configuration files in the docker
directory.
Upgrade Guide
Docker compose deployments
Warning
The docker-compose.yaml
has been refactored. If you've made any changes to the file, make sure to check out the "Upgrade to new docker compose deployment" section above for usage and migration tips.
-
Back up your customized docker-compose YAML file (optional)
cd docker cp docker-compose.yaml docker-compose.yaml.$(date +%s).bak
-
Get the latest code from the main branch
git checkout main git pull origin main
-
Stop the service,Command, please execute in the docker directory
docker compose down
-
Back up data
tar -cvf volumes-$(date +%s).tgz volumes
-
Upgrade services
docker compose up -d
Source Code deployments
-
Stop API server, Worker and Web frontend Server.
-
Get the latest code from the main branch:
git checkout main git pull origin main
-
Update Python dependencies:
cd api poetry install
-
Then, let's run the migration script:
poetry shell flask db upgrade
-
Finally, run API server, Worker and Web frontend Server again.
What's Changed
- fix: wrong order of history prompts in ReAct agent mode by @sinomoe in #5236
- fix: embedding job fails using IAM role by @tmokmss in #5252
- feat(Tools): Add Serply Web/Job/Scholar/News Search tool for more options by @googio in #5186
- feat: Added hindi translation i18n by @Yash-1511 in #5240
- fix: correct typos in the icons for microsoft by @kurokobo in #5243
- fix: modal z-index cleanup by @chazzhou in #5234
- chore: add icon in .idea by @Gallardot in #5259
- fix: run agent with Vertex AI Gemini models by @hellof20 in #5260
- fix: respect the interface language specified by the user on the activation success screen by @kurokobo in #5258
- fix: visioin model always with low quality by @rerorero in #5253
- fix: casting non-string type value for tool parameter options by @bowenliang123 in #5267
- Feat: support delete account by @JzoNgKVO in #5208
- fix: copyright with latest time by @crazywoola in #5271
- Fix: z-index of delete account modal by @JzoNgKVO in #5277
- Fixed wrong /text-to-audio curl example by @from2001 in #5286
- fix: add event handler to delete the site when the related app deleted by @kurokobo in #5282
- fix: typo and check by @crazywoola in #5287
- Dalle3 add seed by @charli117 in #5288
- fix(core): Reorder
field_validator
andclassmethod
to fit Pydantic V2. by @laipz8200 in #5257 - fix: not checked require_summary of duckduckgo search raise error by @hjlarry in #5303
- Fix: workflow result display by @JzoNgKVO in #5299
- chore: upgrade next to 14.1.1 by @iamjoel in #5310
- feat: permission and security fixes by @chazzhou in #5266
- fix: unable to reindex documents by @sinomoe in #5276
- fix:update Member field error by @DomKing in #5295
- fix: pin tenacity to 8.3.0 by @bowenliang123 in #5319
- feat: support tencent cos storage by @quicksandznzn in #5297
- add the filename length limit by @JohnJyong in #5326
- Add bedrock command r models by @realjustinwu in #4521
- fix(core/workflow): Handle special values in node run result outputs by @laipz8200 in #5321
- chore(core/workflow/utils/variable_template_parser): Refactor VariableTemplateParser class for better readability and maintainability. by @laipz8200 in #5328
- fix: got unknown type of prompt message in multi-round ReAct agent chat by @sinomoe in #5245
- fix: allow special characters in email by @yamitzky in #5327
- Update and fix the model param of Deepseek by @richards199999 in #5329
- fix: extract params by function calling for models supporting tool call by @sinomoe in #5334
- chore: update llm.py by @eltociear in #5335
- fix: workflow results in FAIL status due to null reference error by @rerorero in #5332
- feat: support Latex by @ZuzooVn in #5001
- feat: add Novita AI image generation tool, implemented model search, text-to-image and create tile functionalities by @XiaoLey in #5308
- fix: wrong token usage in iteration node for streaming result by @rerorero in #5336
- feat: add
flask upgrade-db
command for running db upgrade with redis lock by @bowenliang123 in #5333 - chore: set build system to Poetry and remove unnecessary settings with package mode disabled by @bowenliang123 in #5263
- dep: bump chromadb from 0.5.0 to 0.5.1 by @bowenliang123 in #5345
- chore: add
novita_client
topyproject.toml
by @laipz8200 in #5349 - fix: resolve issue with cot_agent_runner not analyzing user-uploaded images correctly by @XiaoLey in #5360
- Update README.md by @horochx in #5359
- docs(readme): Optimize the content in the readme file by @beita1 in #5364
- docs: Add notes for running Postgres Docker on Windows WSL2 Ubuntu … by @hikariming in #5373
- feat: allow non-english wikipedias to be searched by @kurokobo in #5371
- fix: remove conversation_id description in completion-messages docs by @huxiaoxi1992 in #5372
- feat: default timezone to user's local timezone in activate form by @chazzhou in #5374
- Fix: multi image preview sign by @fishisnow in #5376
- Add sample environment variables for Aliyun OSS by @denverdino in #5366
- feat: support opensearch approximate k-NN by @baojingyu in #5322
- feat: introduce pydantic-settings for config definition and validation by @bowenliang123 in #5202
- feat: initial support for Milvus 2.4.x by @bowenliang123 in #3795
- refactor: refactor the button component using
forwardRef
by @TinsFox in #4379 - Fix: use new button by @JzoNgKVO in #5384
- docs(api/README): Remove unnecessary
=
by @laipz8200 in #5380 - feat: add log date timezone by @liuzhenghua in #4623
- chore: extract retrival method literal values into enum by @bowenliang123 in #5060
- fix: unnecessory data fetch when swithing apps category on explore page by @doufa in #5155
- feat: update template by @crazywoola in #5395
- Corrected an error in the APi docs by @Theysua in #5398
- fix bug: tencent vdb #5378 by @quicksandznzn in #5408
- feat: new icons by @zxhlyh in #5412
- fix: optional parameter missing default value None in http request node by @takatost in #5413
- fix: Revert "feat: initial support for Milvus 2.4.x (#3795)" downgrading to 2.3.x for Linux arm64 installation failure by @bowenliang123 in #5414
- fix: prompt editor insert cursor position by @zxhlyh in #5415
- fix: sentry config float type err by @takatost in #5416
- fix: button component will refresh page by @iamjoel in #5420
- fix: workflow note node copy & link style by @zxhlyh in #5428
- fix(core): Fix incorrect type hints. by @laipz8200 in #5427
- fix: annotation id not pass to update setting by @iamjoel in #5429
- fix: add notion page in knowledge by @zxhlyh in #5430
- improve: unify Excel files parsing in either xls or xlsx file format by Pandas by @bowenliang123 in #4965
- refactor: Remove unused code in large_language_model.py by @laipz8200 in #5433
- refactor: extract vdb configs into pydantic-setting based dify configs by @bowenliang123 in #5426
- fix: in iteration node picker may show the wrong var type by @iamjoel in #5435
- fix: some types of buttons ui breaks by @iamjoel in #5437
- fix/i18n: correct
indexMethodHighQualityTip
by @GOODBOY008 in #5431 - feat: add support for claude-3-5-sonnet-20240620 by @takatost in #5452
- feat: add support for bedrock claude-3-5-sonnet-20240620 by @soulteary in #5461
- feat(api/auth): switch-to-stateful-authentication by @laipz8200 in #5438
- feat: option to hide workflow steps by @crazywoola in #5436
- fix: button by @zxhlyh in #5470
- fix: view workflow log detail page crash by @iamjoel in #5474
- feat: add support for Vertex AI claude-3-5-sonnet@20240620 by @hellof20 in #5475
- fix: max_tokens of qwen-plus & qwen-plus-chat by @crazywoola in #5480
- build: support Poetry for depencencies tool in api's Dockerfile by @bowenliang123 in #5105
- Add Oracle23ai as a vector datasource by @tmuife in #5342
- chore: remove pip support for api service by @takatost in #5453
- chore: use singular style in config class name by @bowenliang123 in #5489
- refactor: extract storage provider configs into dify configs by @bowenliang123 in #5443
- refactor: extract db configs and celery configs into dify config by @bowenliang123 in #5491
- fix: firecrawl apikey not start with fc- by @iamjoel in #5498
- fix: tongyi json output by @lxm in #5396
- fix: zhipu tool calling, this PR fixes the bug described in issue #5496 by @vccler in #5469
- fix: summary of duckduckgo_search by @hjlarry in #5488
- refactor: extract hosted service configs into dify config by @bowenliang123 in #5504
- chore: use singular style in middleware config class name by @bowenliang123 in #5502
- fix: correct typos by @kurokobo in #5510
- feat: add Asia/Ho_Chi_Minh timezone by @ZuzooVn in #5521
- fix: added error handling for novita ai tool query by @XiaoLey in #5506
- fix: apply best practices for the latest buildkit by @kurokobo in #5527
- feat: add new features to enhance image and link handling in Jina tool by @XiaoLey in #5517
- feat: use root dir to start python and celery by @crazywoola in #5515
- Chore: chat log refactor by @JzoNgKVO in #5523
- Fix: custom disclaim by @JzoNgKVO in #5535
- fix: resolve issue with embedding model field visibility toggling on datasets page by @GOODBOY008 in #5451
- feat: make Citations and Attributions display enable default by @takatost in #5508
- add opensearch default value by @JohnJyong in #5536
- feat: Add program_name attribute to TiDB connection by @sunxiaoguang in #5499
- Fix: position of log modal by @JzoNgKVO in #5538
- fix(api/model_runtime/azure/llm): Switch to tool_call. by @laipz8200 in #5541
- chore: add create_json_message api for tools by @hjlarry in #5440
- chore: refactor the http executor node by @hjlarry in #5212
- feat: support predefined models for openrouter by @sinomoe in #5494
- fix: fe tool filter missing different languages handling by @Yeuoly in #5558
- fix: useless CI style checks by @Yeuoly in #5559
- fix: revert CI path filters by @Yeuoly in #5561
- The firecrawl tool now supports self-hosting by @keita69 in #5528
- feat: support importing and overwriting workflow DSL by @takatost in #5511
- refactor: extract cors configs into dify config and cleanup the config class by @bowenliang123 in #5507
- Fix/single run panel show parent scrollbar by @aixgeek in #5574
- fix: add support for FILE type in ToolParameterConverter by @laipz8200 in #5578
- chore: fix typo in config descriptions by @bowenliang123 in #5585
- feat: update issue template by @crazywoola in #5592
- fix: Modify the incorrect configuration name for Google storage by @hellof20 in #5595
- fix: context icon in chat by @zxhlyh in #5604
- delete the deprecated method by @JohnJyong in #5612
- fix: DuckDuckGo image search tool error by @hjlarry in #5606
- fix: populate app configs to system environment variables by @bowenliang123 in #5590
- feat: undo/redo for workflow editor by @perzeuss in #3927
- Fix link to documentation of nodes by @johnpccd in #5623
- chore(pyproject.toml): Add Ruff formatter config. by @laipz8200 in #5627
- chore: cleanup test_delete_by_document_id method in opensearch vdb test by @bowenliang123 in #5619
- feat: add ops trace by @ZhouhaoJiang in #5483
- feat: tracing fe by @iamjoel in #5487
- FR: #4048 - Add color customization to the chatbot by @dromerolovo in #4885
- fix: ro-RO is not a valid language by @hjlarry in #5635
- chore: add a secondary confirmation dialog when the user delete the tool by @hjlarry in #5634
- fix: remove obsoleted 'version' elements in compose files by @kurokobo in #5553
- fix: type error in config by @takatost in #5643
- fix(api/configs): Ignore empty environment variables when loading config. by @laipz8200 in #5647
- fix(api): language list by @luckylhb90 in #5649
- chore: apply flake8-comprehensions Ruff rules to improve collection comprehensions by @bowenliang123 in #5652
- fix: HTTP request header is overwritten when user set Content-Type by @hjlarry in #5628
- fix: tool call message role according to credentials by @sunxichen in #5625
- feat: add jina new pre-defined rerankers, include: jina-reranker-v2 by @takatost in #5657
- fix: knowledge retrieval score threshold setting by @zxhlyh in #5658
- chore: delete unused resource by @ZuzooVn in #5667
- chore: rearrange python dependencies in groups by @bowenliang123 in #5603
- feat: xxo enhancement. by @GarfieldDai in #5671
- Feat/fix ops trace by @ZhouhaoJiang in #5672
- feature: Add presence_penalty and frequency_penalty parameters to the … by @liuzhenghua in #5637
- Fix docker command by @WilliamEspegren in #5681
- Feature/add qwen llm by @LiXuemin in #5659
- fix: do not remove by @luckylhb90 in #5682
- Feat/add json process tool by @scotlandowl in #5555
- feat: change TRACE_QUEUE_MANAGER_INTERVAL default value by @ZhouhaoJiang in #5698
- Chore/improve deployment flow by @guchenhe in #4299
- Correction of Typo in French by @thibautleaux-kreactive in #5699
- feat: add support Spark4.0 by @leslie2046 in #5688
- fix: yarn lock file missing by @takatost in #5703
- build(deps): bump braces from 3.0.2 to 3.0.3 in /web by @dependabot in #5705
- fix: do not remove by @takatost in #5706
- add docker-legacy and docker/nginx/conf.d/default.conf to .gitignore by @guchenhe in #5707
- feat: add fix-app-site-missing command by @takatost in #5711
- fix: fix-app-site-missing command by @takatost in #5714
- fix: _convert_prompt_message_to_dict parameters err by @takatost in #5716
- Ignore new middleware.env docker file by @WilliamEspegren in #5715
- bump to 0.6.12 by @takatost in #5712
New Contributors
- @googio made their first contribution in #5186
- @Gallardot made their first contribution in #5259
- @from2001 made their first contribution in #5286
- @yamitzky made their first contribution in #5327
- @horochx made their first contribution in #5359
- @beita1 made their first contribution in #5364
- @huxiaoxi1992 made their first contribution in #5372
- @baojingyu made their first contribution in #5322
- @tmuife made their first contribution in #5342
- @lxm made their first contribution in #5396
- @vccler made their first contribution in #5469
- @sunxiaoguang made their first contribution in #5499
- @keita69 made their first contribution in #5528
- @aixgeek made their first contribution in #5574
- @dromerolovo made their first contribution in #4885
- @luckylhb90 made their first contribution in #5649
- @sunxichen made their first contribution in #5625
- @WilliamEspegren made their first contribution in #5681
- @LiXuemin made their first contribution in #5659
- @scotlandowl made their first contribution in #5555
- @thibautleaux-kreactive made their first contribution in #5699
Full Changelog: 0.6.11...0.6.12