agent0ai/agent-zero v2.11

on GitHub

Release Notes

This release introduces several new bundled plugins, significantly improves streaming and real-time UI performance, adds vision sidecar integration with model presets, and delivers broad reliability fixes across memory, embeddings, context handling, and the interactive browser.

New Plugins & Extensions

• Context Window Usage Indicator — A bundled plugin showing a composer usage ring, responsive breakdown popover with token counts, percentages, free-space reporting, and provider-reported price/cache hit details. Includes independent mobile and desktop visibility controls.
• Context Doctor — Automatically repairs malformed tool-call JSON before dispatch, handles incomplete or thoughts-only responses, supports XML suppression, and persists repaired output in WebUI logs. Includes reasoning key-value pair handling and configurable log-detail settings.
• Vision Sidecar Integration — Strict per-preset sidecar slot with Main-first routing, one-call multi-image analysis, parallel-worker image resolution, configurable timeout/max-token controls, and inline model settings. Native vision output from parallel workers is preserved and promoted correctly.
• Result Extension Point — Extensions can now normalize or consume completed model results before default history and tool processing. Repeat detection and empty-response handling are split into dedicated, configurable loop-control extensions with prompt-backed retry limits.
• WebUI Plugin Slots — New extension slots beside the model/profile strip and at the end of interface visibility controls, allowing plugins to register responsive UI affordances.

Streaming & Real-Time Performance

• Responses now render with safe Markdown and LaTeX throughout live streaming, removing the raw-to-formatted swap at completion.
• Native response tool output (e.g., Codex answers) streams progressively through the canonical tool envelope.
• Expensive full-response parsing is coalesced behind a 128-character or 50-millisecond cadence while retaining per-chunk security hooks and strict tool validation.
• Message action buttons remain stable during streamed updates, preserving hover, focus, and extension controls.
• Native reasoning summaries are promoted as thoughts after streaming completes.
• Incomplete stream roots are rejected before invoking the dirty JSON scanner, avoiding repeated scans of partial prefixes.

WebUI & Synchronization

• Optimized real-time WebUI synchronization with collection deltas, stable sidebar row identity, bounded streamed log fields, deferred response formatting, and capped process group updates.
• Chat list state is preserved when filtered/sorted snapshots are byte-equivalent, with proper selection and tree synchronization.
• Branched chats detach provider state cleanly — inherited Responses continuation, response-ID ownership, and cached context data are rebuilt from trimmed local history.
• Context window indicator refreshes when each new generation starts, with deduplicated streamed updates.
• Step detail header actions are aligned with consistent height and mobile-friendly wrapping.
• First user turn renders optimistically; WebSocket state pushes are serialized so welcome-screen messages remain visible.
• Credential fields render as masked text inputs instead of password fields, preventing browser password-save prompts.

Embeddings & Memory

• Gateway model IDs (e.g., nvidia/llama-nemotron-embed-vl-1b-v2:free, openrouter/openai/text-embedding-3-small) are now kept intact for OpenAI-compatible embeddings — the provider prefix is always applied, matching chat wrapper behavior.
• NumPy scalar relevance scores are coerced to native floats at the source, fixing float32 is not JSON serializable crashes. Legacy NumPy scores are converted only in the memory dashboard response, keeping shared API serialization strict.
• Preloaded local SentenceTransformer models are cached by constructor options, so startup preload and first-chat initialization share one instance.
• Bundled memory operations are now reliable across prompt objects without crossing profile, project, or memory-store scope, with a single query embedding reused across memory and solution searches.

Context & Prompt Handling

• Compacted context summaries are stored as non-assistant context so provider normalization no longer discards them as orphaned assistant turns.
• Settings snapshots are scoped to prompt builds, avoiding repeated Git and default resolution while keeping the canonical settings function unchanged.
• Protocol and extras context JSON, parallel job results, and started-job handles are serialized without whitespace to reduce prompt tokens.
• Inherited tool-policy prompts are returned unchanged before inventory resolution; custom policies keep the existing filtering path.
• Automatic chat naming is deferred to monologue_end so it cannot delay the Main Model's first token.

Model & Provider

• Provider-reported token usage and response cost are carried through Chat Completions and Responses, including terminal streaming usage chunks. Missing provider accounting stays absent rather than synthesized.
• Non-ASCII function-call arguments are preserved in Responses API output; automated structured user turns use compact JSON.
• Default model transport is now Chat Completions — OpenAI, Azure, GitHub Copilot, and GitHub Copilot OAuth route through Chat Completions explicitly. Codex/ChatGPT and xAI Grok OAuth remain on Responses. New providers default to Chat Completions unless they explicitly opt in to Responses.

Interactive Browser

• Configurable keyboard layouts — New keyboard_layout and keyboard_variant browser settings apply the chosen XKB layout to the private display and pin it on the Xpra shadow server. Non-US keyboards (German Mac @ via Option+L, umlauts, AltGr symbols, etc.) now work correctly in the interactive browser window.
• Fallback canvas input forwards text from AltGraph/macOS Option input while preserving ordinary Windows/Linux Alt shortcuts.
• Stable browser IDs are preferred in Browser Settings; exact advertised legacy CDP endpoints are translated before connector dispatch.

Fixes

• Mobile canvas rail can now be properly hidden via Alpine's x-show state.
• Onboarding modal closes before opening model presets.
• Empty vision skip summaries are hidden when no images were skipped.
• ACP plugin thumbnail restored with original artwork.
• Launcher README links updated to v1.6.
• Context Doctor dependency (json_repair) is installed automatically after self-update without requiring an image rebuild.

Breaking Changes

• Default LiteLLM transport changed to Chat Completions. Providers that previously relied on the implicit Responses transport must now explicitly set their transport mode to Responses. OpenAI, Azure, and GitHub Copilot providers are unaffected as they are explicitly configured.