github Mintplex-Labs/anything-llm v1.16.1
AnythingLLM v1.16.1
on GitHub

3 hours ago

Activity and Chain of Thought Overhaul

image

We overhauled our UI for long-horizon agentic tasks that show multiple thinking and tool calls to now roll up into a single clean, collapsible UI element.

This gives you more screen real estate for the actual response while still keeping the important call stack and thoughts the model produces in an easy-to-read format.

Navigation Warning

image

Recently, we added true abort signals across every supported LLM so that when you cancel inference or leave the page, we stop inference to save you on compute costs.

However, this had the unintended side effect of canceling inference when navigating away from a chat, preventing background completion. Now we warn you before aborting the response.

Roadmap item: We intend to make this user configurable so you can customize this behavior and allow background inference for those with more powerful or high-concurrency setups where multiple parallel inference sessions are feasible.

Foundry Local Embedded (Desktop Only)

image

FoundryLocal is an inference engine built by Microsoft that is pre-installed and available on Windows 11 (version 24H2, build 26100 or later).

This uses WinML to load models for your CPU, GPU, or NPU across any Windows hardware configuration. It's a great option for hardware configurations we don't have explicit support for. Keep in mind that model availability may be limited as FoundryLocal is still rolling out. Vision models are not available currently.

We partner with Microsoft for this, if you have bugs or issues we can help forward them to the FoundryLocal team

GenieX for Snapdragon (Desktop Only)

image

Note

This replaces our previous Snapdragon NPU engine. Any previous models from the old engine are deleted on update, and you'll need to re-download these optimized models. You should see a significant performance increase for NPU models and regular GGUFs.

We highly recommend using only GGUF models for this engine. Qualcomm NPU-only models have 4K context windows and will result in a poor agent experience.

GenieX is an open-source inference engine built by Qualcomm (previously NexaAI) that efficiently runs GGUFs and Qualcomm AI Hub models on NPU for X and X2 Elite devices.

AnythingLLM offers fully managed and built-in support by selecting the "AnythingLLM GenieX" provider in the dropdown. Both GGUFs and NPU-only models can leverage the NPU, giving you the power efficiency and intelligence of both.

We partner with Qualcomm for this, if you have bugs or issues we can help forward them to the GenieX team or you can make an issue on their GitHub. s/o @zhiyuan8, @alexchen4ai & @alanzhuly

New Features

  • Native Foundry Local SDK on Windows (x64/ARM64) - the Foundry provider is now fully self-contained and no longer requires a separate Foundry Local install, with GPU/NPU-optimized model variants exposed in the catalog
  • GenieX runtime support for Windows ARM64 devices
  • AWS Bedrock cross-region inference profile support - geo-prefixed profile IDs (us., eu., global.) now appear in the model dropdown and route correctly, plus manual region entry for regions outside the preset list (GovCloud, specialized partitions)
  • LocalAI added as an image generation provider, listing only image-capable models from your server
  • LocalAI context window auto-detection - context windows are read per-model from your server's config, with the manual setting now an optional override
  • Silent/headless install and uninstall flags for the Windows installer - see the Windows installation docs

Improvements

  • Chain of thought and agent activity are now a single collapsible component - one line when collapsed, one step per thought with a connected rail when expanded, and the live thought shown in the header while streaming
  • The new chain of thought rollup is now mirrored in the Assistant Panel
  • Leaving a thread or navigating away mid-inference now warns you before the response is lost, instead of silently killing the generation
  • Foundry Local no longer prompts for setup when using the built-in runtime, and auto-loads models by capability detection
  • Onboarding light mode styling and a redesigned LLM selection UI
  • Beacon now renders thinking blocks and markdown formatting correctly while streaming, with collapsible thinking and a fix for model detail text overflow
  • Windows installer now includes a sidebar panel image
  • Prompt input drafts now persist between navigation

Bug Fixes

  • Fixed a Gemini crash caused by a content-header length mismatch on unpinned undici versions - now compatible with both v6 and v7
  • Fixed a crash when aborting an Anthropic (or Bedrock Anthropic) response mid-stream in chat mode and then sending a follow-up
  • Fixed a memory leak from leaked abort-stream listeners, which also caused pausing a response to wipe the entire chat
  • Fixed a crash when creating a new workspace
  • Fixed missing setup CTAs on modals introduced by the new uniform modal component - Scheduled Job skill setup on both the category and tool views, Community Hub connection key, and the experimental features reject button
  • Fixed the intermediate loader being too tall while waiting for an agent capability response
  • Fixed the stray border around the delete button in Workspace Chats
  • Fixed reasoning getting stuck on "thinking" when a model never closes its think tag
  • Fixed agent websocket sends firing while the connection was still opening

All Changes

New Contributors

Full Changelog: v1.16.0...v1.16.1

Check out latest releases or
releases around Mintplex-Labs/anything-llm v1.16.1

Don't miss a new anything-llm release

NewReleases is sending notifications on new releases.

Get notifications