BerriAI/litellm v1.111.0-dev.4 on GitHub

Important

Signing has moved to keyless cosign. Since v1.111.0-dev.3, releases are signed with Sigstore keyless signing (GitHub OIDC) instead of the cosign.pub key.
cosign verify --key ... does not work for this release. Use the command below. Earlier releases still verify with the key.

Verify this release

cosign verify \
  --certificate-identity https://github.com/BerriAI/project-releaser/.github/workflows/release.yml@refs/heads/main \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/berriai/litellm:v1.111.0-dev.4

Same command for docker.io/litellm/litellm and the Helm chart at ghcr.io/berriai/litellm/chart/litellm:1.111.0-dev.4

What's new in 1.111.0-dev.4

138 PRs. 18 added, 69 fixed, 11 changed, 3 improved, 1 removed. 2 need action

Action required

Logging and observability: Set general_settings.redact_tokens_in_spend_logs: true to keep token scanning in spend logs (#51729)
AWS Bedrock: Remove unsupported service_tier values from Bedrock requests (#50516)

Highlights

Admin UI: Added customer organization, team and member management in Customer Management (#51521, #51661, #51713)
Vector stores: Added OpenSearch vector store filters support and hybrid keyword plus vector search (#51131)
Keys, teams and auth: Added budget_limits windows on internal users (#51763)
Keys, teams and auth: Added email warnings to key owners before opted-in virtual keys expire (#51482)
SDK: Fixed hang and RecursionError on every completion call when litellm.model_fallbacks is set (#51817)
Deployment: Fixed proxy boot failure on PostgreSQL 12 without pgcrypto, broken since v1.110.0-rc.1 (#51868)
Performance: Improved streaming latency by closing streams right after the last event (#51487)
Responses API: Fixed Responses fallback replaying tool calls that were already delivered (#51683)
Azure: Fixed 400 on azure/ Responses and chat calls to Foundry project /openai/v1 bases (#51767)

All changes

OpenAI (2)

Fixed batch prices for gpt-image-2.6-sunburst and gpt-image-2.6-flare, billed at twice the rate before (#51647)
Fixed 500 and unbilled first round on web search with openai/responses/ chat completions (#51802)

Azure (5)

Fixed retirement dates for azure_ai/claude-haiku-4-5 and azure_ai/claude-opus-4-5 to 2027-03-16 (#51716)
Fixed reasoning_effort 400 and dropped reasoning_content for Kimi K3 on azure_ai (#51747)
Fixed server tracebacks reaching clients and alerts when an Azure stream is cut (#51760)
Fixed 400 on azure/ Responses and chat calls to Foundry project /openai/v1 bases (#51767)
Changed /v1/messages for Claude on Azure AI Foundry to run on the Rust gateway (#51721)

AWS Bedrock (5)

Fixed 500 on Bedrock Converse requests with s3:// image, document or video URLs (#50577)
Fixed aborted Bedrock Invoke /v1/messages streams still reading from Bedrock (#50657)
Fixed Bedrock vector store search 400 on OpenAI-shape filters and ignored drop_params (#51291)
Fixed Titan embedding calls on Bedrock /invoke passthrough not being billed or logged (#51701)
Changed Bedrock to return 400 for unsupported service_tier values and to forward reserved (#50516)

Vertex AI and Gemini (9)

Added key, team, org and project labels on Vertex AI requests (#50732)
Fixed wrong default max_tokens for versioned Claude ids on Vertex AI (#40376)
Fixed shutdown dates on Gemini TTS and Live preview models to match Google's deprecations page (#51657)
Fixed PDF embeddings on gemini-embedding-2 reporting 0 tokens and $0 cost (#51673)
Fixed document_ocr, audio_track_extraction and auto_truncate being dropped on Gemini embeddings (#51686)
Fixed streamed Gemini TTS on chat completions returning no audio in delta.audio (#51750)
Fixed Vertex 400 for tools with a top-level anyOf, oneOf or allOf schema (#51835)
Improved Vertex RAG Engine vector store searches by reusing one Google token (#51239)
Removed Flex prices from three Vertex AI Gemini 4 rows (#51735)

Other providers (13)

Added Vercel AI Gateway models for Decisions requests with cost tracking (#50642)
Added GitHub App authentication with PEM upload for GitHub Copilot (#51427)
Added fal.ai Pixverse video billing by clip duration and resolution, billed at $0 before (#51679)
Added github_copilot/gemini-4-hydrogen to the model cost map (#51696)
Added the 4K output price for runwayml/seedance2_5, billed at the 720p rate before (#51697)
Fixed OpenRouter streams emitting an empty choice after finish_reason: stop (#50210)
Fixed Cohere embeddings ignoring COHERE_API_BASE (#50249)
Fixed Hugging Face chat ignoring HUGGINGFACE_API_BASE and crashing with env endpoints and huggingface/tgi (#51449)
Fixed low and medium reasoning effort being sent as high for mistral/mistral-large-4 (#51693)
Fixed 500 when streaming opencode_go Messages-protocol models through chat completions (#51715)
Fixed together_ai/* listing retired Together models that answer 400 (#51827)
Changed OpenRouter failed non-streaming completions to raise instead of returning success (#50880)
Changed Mistral Magistral to receive native reasoning levels instead of prompt instructions (#51329)

Chat completions (1)

Fixed 500 on gateway chat completions when Claude returns a thinking block (#51774)

Responses API (2)

Fixed Responses fallback replaying tool calls that were already delivered (#51683)
Fixed 500 when allowed_openai_params is a string or tuple on Responses models, broken since v1.109.0-rc.1 (#51751)

Messages API (2)

Fixed synchronous iteration of streamed /v1/messages calls routed to OpenAI or Azure (#51305)
Fixed gateway ignoring a deployment's thinking, system and tools params on Messages requests (#51834)

Batches and files (2)

Added per-model enqueued token limits for batches on projects (#51608)
Fixed 404 retrieving or deleting a managed file when the request also names a model (#51780)

Vector stores (1)

Added OpenSearch vector store filters support and hybrid keyword plus vector search (#51131)

Token counting (4)

Added native upstream token counting for Mistral (#50653)
Fixed token counts ignoring thinking, reasoning effort and tool_choice (#51079)
Fixed local token counts ignoring the request's tool_choice on the count routes (#51133)
Changed /v1/messages/count_tokens to answer 400 instead of 500 for messages missing role or content (#51284)

MCP (2)

Fixed MCP gateway sign-in ignoring the servers selected on the connect page (#50973)
Fixed resources/read failing on the aggregate /mcp endpoint with two or more servers (#51462)

Guardrails (1)

Changed Straiker blocked prompts to return 200 with the block message instead of 400 (#51401)

Caching (2)

Fixed in-memory cache keeping expired keys and panicking on very long TTLs (#51388)
Fixed cache_control being dropped on tool plus reasoning chat requests sent over Responses (#51783)

Routing and fallbacks (4)

Fixed Complexity Router dropping temperature and output-token limits from routed prompts (#51702)
Fixed /model_group/info missing key aliases and team model aliases (#51754)
Fixed wildcard aliases over unknown orgs listing every catalog model id (#51805)
Changed over-tpm requests to fail at once with an error naming the request size (#51838)

Cost tracking and pricing (3)

Added cost map rows for Soniox v6 and four DeepInfra models, and corrected retirement dates (#51620)
Added fal_ai Seedream v5 Pro US text-to-image and edit pricing (#51881)
Fixed cost maps with a byte order mark failing to load and register_model(url) calling out in offline mode (#51545)

Logging and observability (8)

Added a proxy-level default langfuse_span_scope for key and team Langfuse destinations (#51726)
Fixed Langfuse environment, release and version missing on langfuse_otel traces with OTel v2 (#51385)
Fixed system prompt blocks sent to /v1/messages missing from logged messages (#51699)
Fixed tool and auto-router usage being lost after database disconnects (#51706)
Fixed missing litellm_dynamic_limiter_share_tokens gauge for uncapped teams served while the model is idle (#51709)
Fixed spend logs dropping caller request metadata on auth failures (#51745)
Fixed duplicate async success logs for sync streamed openai/responses/ chat calls (#51787)
Changed spend log token redaction to be off by default (#51729)

Keys, teams and auth (18)

Added email warnings to key owners before opted-in virtual keys expire (#51482)
Added budget_limits windows on internal users (#51763)
Fixed carried end-user spend being lost when a budget reset is retried (#51143)
Fixed 401 Expired Key in Claude Code after a lite login --pkce --config-claude key expires, broken since v1.102.0 (#51319)
Fixed customer org provisioning retries failing and duplicate org aliases breaking requests (#51607)
Fixed customer budget alerts not firing for members and ignoring team budget overrides (#51645)
Fixed budget updates failing for customer teams and stale member caches after inherited budget changes (#51654)
Fixed first request for a new customer org dropping its member, so member spend was never tracked (#51660)
Fixed dashboard sessions ignoring team models, which made Aria show No available models (#51663)
Fixed budget_override_expires_at being stored on customer rows with no budget_override (#51667)
Fixed budgets set as 12months or 1month resetting every midnight instead of on the calendar schedule (#51672)
Fixed a key pinned to one customer org registering a different org alias before its 403 (#51674)
Fixed /customer/new budgets resetting on a different schedule than other budget endpoints (#51689)
Fixed ExceededBudget persisting after a customer budget reset (#51694)
Fixed 403 for existing customer members on retry or alias update at the free tier team limit (#51698)
Fixed leftover budget rows when deleting a customer team that only had budget_override (#51703)
Fixed /queue/chat/completions skipping budget checks and request_tag_mappings (#51753)
Fixed SCIM PATCH or PUT 404 on a group with no team so user deactivation applies, broken since v1.99.0 (#51829)

Admin UI (7)

Added customer organization, team and member management in Customer Management (#51521, #51661, #51713)
Added an admin setting to turn Aria on or off for internal users (#51641)
Added free tier limit of 5 customer organizations, teams and members without an Enterprise license (#51687)
Fixed customer budget displays showing No cap for monthly and yearly budgets (#51658)
Fixed customer org settings tab rendering in a narrow left column (#51700)
Changed customer usage page to show member spend above top models (#51677)
Changed Customer Portal to Customer Management in the UI and API error titles (#51891)

Deployment (4)

Fixed Helm Ingress sending 16 gateway-only paths to backend pods instead of the gateway (#50760)
Fixed Docker image reporting a stale litellm-proxy package version, broken since v1.110.0-dev.2 (#51807)
Fixed proxy boot failure on PostgreSQL 12 without pgcrypto, broken since v1.110.0-rc.1 (#51868)
Changed the Docker base image to a Wolfi build with patched glibc (#51550)

SDK (2)

Fixed missing /v1/messages streaming callbacks on SDK installs without redis, broken since v1.98.0 (#51685)
Fixed hang and RecursionError on every completion call when litellm.model_fallbacks is set (#51817)

Performance (2)

Improved streaming latency by closing streams right after the last event (#51487)
Improved spend log token redaction speed for large prompts (#51770)

Other (3)

Added native generateContent serving for custom providers (#51083)
Fixed cancel_on_disconnect not applying to /v1/rerank (#51252)
Fixed AttributeError on parallel first requests to a fresh worker (#51591)

Full changelog