Built-in LLM for Docker
AnythingLLM Docker now has a default local LLM experience right out of the box. The same kind of built-in engine that ships with AnythingLLM Desktop can now run inside the container: Lemonade, the fully open-source local inference server from AMD, running your models with llama.cpp. You no longer need to run Ollama, LM Studio or any other inference server next to AnythingLLM.
Pass your GPU into the container and the engine is accelerated on whatever hardware you have. It picks the fastest backend automatically: CUDA for NVIDIA GPUs, Vulkan for AMD and Intel GPUs, or the CPU. It runs on both amd64 and arm64 hosts.
When it's turned on, Built-in becomes the default LLM provider during onboarding, with three models recommended for your hardware. You can download any model from the model table or pull any GGUF model from Hugging Face. It works everywhere a provider can be used: the system LLM, workspace and agent LLMs, the chat window model picker, and Model Router.
Tip
The built-in LLM is off by default. Turn it on by starting the container with -e ENABLE_BUILT_IN_LLM=true, and add --gpus all (or pass in your AMD or Intel GPU devices) for GPU acceleration. See Built-in LLM for Docker for the full setup for each kind of hardware.
Color palettes
Color palettes are straightforward thematic tinting you can apply to the entire UI. We plan to expand this functionality more so you can more heavily customize the UI and even layout. Learn more
| 
|
| 
|
What's Changed
- fix: route Bedrock India (in.) inference profiles via bedrock-runtime by @kimnamu in #6717
- Remove open-computer (moved to its own repo) by @timothycarambat in #6727
- Neutralize spreadsheet formulas in CSV chat exports by @timothycarambat in #6728
- Cap request body size to prevent unauthenticated embed DoS by @timothycarambat in #6729
- feat: show notice when response hits max output token limit (#6703) by @ManojDevarajulu in #6726
- Hold clarifying-question timeout once the user starts answering by @timothycarambat in #6730
- Move Weaviate to weaviate-client v3 by @shatfield4 in #6675
- Support .mov files for transcription by @shatfield4 in #6721
- Stop system prompt variable values from expanding other placeholders by @shatfield4 in #6731
- Apply workspace model changes to agent sessions mid-thread by @shatfield4 in #6673
- Show per-file embed progress in context overflow modal by @shatfield4 in #6639
- fix: include email body/recipients and job prompt in tool approval requests by @rtmalikian in #6732
- fix: send the full GitLab branch name when fetching repository files by @MohammadHijjawi97 in #6740
- fix: return MCP tool text without the result envelope by @Sairaj-24 in #6736
- Fix prompt compressor crash on history records skipped by convertToPromptHistory (#6737) by @ManojDevarajulu in #6738
- fix: keep current-turn attachments when the string compressor runs by @devYRPauli in #6742
- Stop sending deprecated sampling and thinking params to Gemini by @shatfield4 in #6733
- fix: count the text of messages with attachments in statsFrom by @devYRPauli in #6745
- Built-in LLM for Docker (embedded_lemonade) by @timothycarambat in #6719
- fix: truncate long citation URLs in the detail modal by @leilei3167 in #6781
- fix: default OpenAI image generation to gpt-image-2.5-flare by @rtmalikian in #6780
- fix: keep conversation history when the OpenAI compatible chat prompt is compressed by @MarMarLabs in #6777
- fix: tell the user why a password reset was refused by @MarMarLabs in #6771
- fix: report empty connector imports as failures by @MarMarLabs in #6775
- fix(telegram): preserve complete agent answers across message chunks by @jianyx1 in #6767
- fix: connect to the document collector on 127.0.0.1 by @leilei3167 in #6766
- chore: drop API key fragments from web search error messages by @u00dxk2 in #6763
- fix: crash on workspace chat settings when saved LLM provider is not in the list yet by @TMelik in #6761
- docs: remove references to removed sqlite:migrate script by @jrlprost in #6759
- fix(collector): read Excel's short date as an ISO date by @L4XB in #6758
- Add Hungarian (hu) translation by @Proxy5 in #6757
- fix: keep a CRLF file's line endings in filesystem-edit-file by @MohammadHijjawi97 in #6756
- Parse PDF and file links in the bulk link scraper by @shatfield4 in #6786
- fix(a11y): announce a reply once it has finished streaming by @opensource-joe in #6752
- fix: use the repository default branch in the GitHub connector by @Enesuygurs in #6800
- fix(paperless-ngx): keep plain-text document content as it is instead of reading it as HTML by @L4XB in #6795
- fix(confluence): keep code blocks as written instead of running them through htmlToText by @L4XB in #6794
- fix: report failed workspace embeds on API document uploads by @Tong-bit-art in #6793
- fix: regenerate and edit should still count toward daily message limit by @jrlprost in #6785
- fix: isolate watched document sync failures by @Tong-bit-art in #6804
- fix(mcp): treat unsupported ping as non-fatal so servers stay usable by @rtmalikian in #6805
- fix: fall back to default chunk overlap when it is not smaller than the chunk size by @RaphaelFakhri in #6811
- fix: use the repository default branch in the GitLab and Gitea connectors by @JoAdo07 in #6809
- fix: preserve daily message quota when deleting threads by @Max-Reisinger in #6814
- fix(ollama): keep context windows when one model show fails by @leilei3167 in #6816
- feat: instance color palettes and tabbed branding settings by @timothycarambat in #6796
- chore: bump version to 1.19.0 by @timothycarambat in #6822
- fix(mistral): support Mistral reasoning models, dedupe model list, add thinking toggle (#6821) by @timothycarambat in #6825
New Contributors
- @ManojDevarajulu made their first contribution in #6726
- @rtmalikian made their first contribution in #6732
- @Sairaj-24 made their first contribution in #6736
- @devYRPauli made their first contribution in #6742
- @jianyx1 made their first contribution in #6767
- @TMelik made their first contribution in #6761
- @jrlprost made their first contribution in #6759
- @Proxy5 made their first contribution in #6757
- @Enesuygurs made their first contribution in #6800
- @Tong-bit-art made their first contribution in #6793
- @RaphaelFakhri made their first contribution in #6811
- @JoAdo07 made their first contribution in #6809
Full Changelog: v1.18.0...v1.19.0