BerriAI/litellm v1.109.0-rc.1 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. This release is signed with the key introduced in commit e82a3c6.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the published signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.109.0-rc.1

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.109.0-rc.1/cosign.pub \
  ghcr.io/berriai/litellm:v1.109.0-rc.1

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

build: allow filelock 4.x and sync package metadata by @tin-berri in #49137
fix(proxy)!: stop answering with the literal string "None" as error type and param on the proxy-server, auth, spend, and key health routes by @mateo-berri in #39540
fix(token_counter): price PDF pages for OpenAI and Azure models the way they bill by @mateo-berri in #47554
fix(ci): measure PR gates from the target branch's current tip by @tavren27 in #49078
fix(wandb): set GLM-5.2 max_input_tokens from the CoreWeave Forge page by @devin-ai-integration[bot] in #49148
fix(streaming): rebuild each choice of an n>1 chat stream on its own by @simonp9 in #49002
fix(batches): return real batch token counts next to OpenAI's batch usage fields by @mateo-berri in #47740
chore(cost-map): add fireworks ultramarine 1 flash us and date deepseek v4 flash by @devin-ai-integration[bot] in #49146
chore(databricks): retirement dates for DeepSeek V4 Flash (0731), V4 Pro (0813) and Inkling by @devin-ai-integration[bot] in #49124
fix(cost-map): set together_ai Qwen3.7-Max back to the published price by @devin-ai-integration[bot] in #49125
refactor(types): replace Any with proven types in 21 files by @norkelbyte in #48921
fix: enforce LiteAdmin MCP route permissions by @tin-berri in #48080
feat(providers): add CLF AI Gateway as an OpenAI-compatible provider with ten models by @bap1106 in #39324
fix(vertex_ai): batch Claude models in Anthropic's native row shape by @Gopal3746 in #45715
fix(cost): bill the requested model's rates when the response names an unpriced model by @norkelbyte in #48900
refactor(helm)!: connect both charts to a separately deployed Lens by @yuneng-berri in #49169
fix(bedrock): backfill published regional service tier prices by @kerry-berri in #49113
feat(cost-map): add baseten joey-1 decision model by @devin-ai-integration[bot] in #49123
fix(cost): bill a provider reported usage.cost on every path unless the deployment is priced by @kerry-berri in #48953
feat(zai): support response_format and max_completion_tokens, add glm-5v-turbo pricing by @DewellNaa in #48335
fix(router): cap the default OTPM reservation and reconcile token counters after each call by @paveln83 in #49054
fix(azure): add batch prices to the unversioned o3, o4-mini, o3-mini, gpt-4o and gpt-4o-mini rows by @devin-ai-integration[bot] in #49182
fix(openai): keep the upstream error body on e.response for OpenAI and Azure errors by @mika-lt in #49079
fix(health): probe only models a partial wildcard serves by @zoveki in #49060
fix(azure_ai)!: keep OpenAI prompt cache breakpoints on the Foundry chat completions path by @devin-ai-integration[bot] in #45299
fix(websearch-interception)!: stream native server-tool blocks to clients by @dkrisman in #40967
fix(count_tokens): count a history that carries a compaction block by @devin-ai-integration[bot] in #44784
fix(proxy): list only routable models for a wildcard alias over a provider org by @mika-lt in #49152
fix(proxy)!: log and bill Responses-style bodies sent to /cursor/chat/completions by @discobot in #30277
fix(mistral): forward prompt_cache_key to the Mistral API by @mateo-berri in #47432
fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply by @mateo-berri in #39864
fix(anthropic): send PDF image_url content to Claude as document blocks by @mateo-berri in #46984
fix(router)!: keep a deployment's own model_id over the request body's by @simonp9 in #49045
fix(proxy): apply new model rows on every worker as soon as they are written by @tessellark in #49034
fix(responses)!: send a dict reasoning_effort the same way on every Responses bridge by @mika-lt in #49082
feat(usage): show subscription-covered providers and account fees separately by @devin-ai-integration[bot] in #44772
fix(router)!: apply content_policy_fallbacks when router settings update at runtime by @devin-ai-integration[bot] in #45428
fix(streaming): convert upstream logprobs so streams with logprobs end with their finish chunk by @devin-ai-integration[bot] in #33549
fix(sso): keep CLI SSO and Claude Code gateway team lists in the user record's order by @zoveki in #49033
fix(cache_control): pick the chat-path dialect from the deployment's provider prefix by @devin-ai-integration[bot] in #45295
fix(bedrock): route a Bedrock model id with no Invoke provider to Converse by @sidhunt in #43290
fix(proxy)!: label shared-handler errors the way the route catch-alls do by @mateo-berri in #39555
fix(mcp): bill both calls of a streamed chat completions MCP auto-execute by @simonp9 in #48899
fix(proxy)!: scope /health by wildcard keys and team model allowlists the way auth does by @zoveki in #49155
fix(router)!: check and count deployment itpm and otpm on every Responses WebSocket turn by @tavren27 in #49069
fix(spend): keep a NaN or infinite cost from wedging daily spend writes by @quietjules in #48829
fix(logging): warn when drop_params drops a requested reasoning level on the remaining paths by @tessellark in #48933
fix(vllm): send the pass-through key as a bearer on both /vllm routes by @alvcol in #46861
fix(vertex_ai,azure_ai)!: auto-add the compaction, thinking display, and tool change betas on Claude routes by @devin-ai-integration[bot] in #44747
fix(passthrough): log the URL model and declared provider on Gemini and Vertex failures by @tessellark in #49075
fix(model_prices): rolling registry audit, add Gemini batch and Azure Astra flex tiers by @devin-ai-integration[bot] in #49114
perf(streaming): stop raising caught AttributeErrors inside pydantic on every streamed chunk by @tavren27 in #49166
fix(gemini): flag computer use support on the four models the Gemini computer use docs list by @devin-ai-integration[bot] in #49252
fix(proxy): evict a deleted key from every worker's auth cache by @etiennechabert in #47609
fix(ui): preserve team model grants on unrelated settings saves by @tin-berri in #49258
fix(aiand)!: show ai&'s name and logo, and sync ai&'s model list by @islomjon-aiand in #49158
docs(daily-activity): describe guardrail spend fields and reporting key IDs in OpenAPI by @mrinal-berri in #49236
fix(responses): normalize empty function_call_output to an empty tool result by @selimfedakar in #48845
test: bedrock claude prompt caching on /v1/messages reports and charges cache tokens by @kerry-berri in #49265
fix(cost): bill the margin once on rebuilt streams and streamed Responses through the chat bridge by @kerry-berri in #49254
refactor(rust): extract SecretValue and use it for Redis credentials by @yujonglee-berri in #49186
chore(cost-map): add openai gpt-6-rosalind from the pricing page and changelog by @devin-ai-integration[bot] in #49268
refactor(ui): remove the deprecated Old Usage page by @kerry-berri in #49284
feat(keys): allow assigning a project to an unassigned key on /key/update by @devin-ai-integration[bot] in #41700
fix(proxy)!: persist sent model_info fields on POST /model/update by @MaximeBOUDIER in #46902
ci: cache Rust compilation with sccache by @narmstrong-berri in #49117
fix(openrouter)!: list OpenRouter's live catalog for openrouter/* when check_provider_endpoint is on by @norkelbyte in #48820
fix(ollama): forward assistant tool call ids in ollama_chat requests by @moe-berri in #49278
chore(cost-map): flag Vertex Gemini computer use and add Cohere Rerank 5 by @devin-ai-integration[bot] in #49297
fix(headroom): forward tenant-scoped session affinity by @tin-berri in #49255
fix(proxy): don't send llm_exceptions alert for BudgetExceededError by @tin-berri in #49290
fix(proxy): apply key and team router_settings to /v1/images/generations by @moe-berri in #49286
feat(openrouter): record upstream provider in logging payload and spend logs by @moe-berri in #49287
fix(proxy): raise live budget counters for failure spend rows by @yucheng-berri in #47741
fix(router): recover streamed context window errors before output by @tin-berri in #49293
test: fix integration tests that fail on test order, races and intended changes by @yuneng-berri in #49310
fix(ui): add Meta Model API to the Add Model provider dropdown by @moe-berri in #49314
fix(rerank): preserve token usage in spend logs by @tin-berri in #49308
fix(exceptions): classify SGLang context window overflow by @neubig in #48837
fix(ui): restore Z.AI credential fields in provider catalog by @moe-berri in #49325
fix(responses): run the guardrails a Responses WebSocket frame selects by @tavren27 in #48725
fix(proxy): preserve access windows when editing models by @tin-berri in #49316
fix(router): budget classifier input by provider limits by @tin-berri in #49306
feat(sso): create the team_id_jwt_field team on UI SSO login when team_id_upsert is set by @jesus-berri in #49256
chore(cost-map): add Deepgram Aura-1 and Aura-2 text-to-speech rates by @shrey-berri in #49280
fix(cost): preserve zero prices across batch, speech and search by @tin-berri in #49294
fix(batches): cost an unmanaged gs:// batch through the deployment that created it by @kerry-berri in #49301
refactor(ui): move policies, tool policies and guardrails monitor text onto i18n by @yuneng-berri in #49343
feat(proxy)!: use Bedrock request metadata as the session id on pass-through routes by @shrey-berri in #49193
fix(gemini): add the batch image output price to gemini-nano-banana-2.1 and gemini-3.1-flash-lite-image by @devin-ai-integration[bot] in #49317
fix(bedrock): align gpt-5.6-cyber Mantle limits and endpoints with its model card by @devin-ai-integration[bot] in #49350
refactor(proxy): give database shutdown one lifespan owner by @yujonglee-berri in #48506
refactor(proxy): inject database repositories through per-repository dependencies by @yujonglee-berri in #48507
refactor(proxy): inject deployment warning ignores through repositories by @yujonglee-berri in #48592
fix(guardrails): name only the request pipeline steps that ran in x-litellm-applied-guardrails by @yucheng-berri in #49262
refactor(proxy): inject gateway activity through a typed repository by @yujonglee-berri in #49116
fix(guardrails): drop the response pipeline steps that never ran from x-litellm-applied-guardrails by @yucheng-berri in #49264
fix(search): forward Perplexity search filters by @tin-berri in #49344
fix(gemini): add the batch, flex and priority text output prices to gemini-2.5-flash-image by @devin-ai-integration[bot] in #49358
fix(proxy): log router-refused image generation requests by @tin-berri in #49349
test(budgets): age the monthly window past any calendar month so the reset test passes on the 31st by @ryan-crabbe-berri in #49372
refactor(router): route the router family through RoutingContext and close the outside surface by @narmstrong-berri in #49118
fix(a2a)!: bill completion-bridge agents at their own prices on /a2a send and stream by @zoveki in #49150
fix(mcp): log SDK request rejections by @mrinal-berri in #49184
test(integration): match Databricks classifier budget telemetry by @mateo-berri in #49382
fix(bedrock): restore the eu.anthropic.claude-haiku-4-5-20251001-v1:0 model map row by @ryan-crabbe-berri in #49375
feat(accounting): add bounded transactional usage persistence by @joshua-berri in #49190
fix(aiohttp): bill the usage a stream sent before its connection dropped, and deflake the CyberArk override test by @mateo-berri in #47838
fix(responses): preserve text verbosity when bridging to chat completions by @ryan-crabbe-berri in #49049
fix(proxy)!: bind admin-owned team keys to their team's MCP servers by @quietjules in #49161
fix(ui): resolve auto router templates through ids a * catch-all lists over a partial wildcard by @tessellark in #49331
feat(passthrough): let a WebSocket relay bill a session from its client text by @shrey-berri in #49281
fix(hosted_vllm): accept the vLLM production-stack router's file and batch objects by @mika-lt in #49068
fix(vertex_ai): bill Claude web search per request on deployments priced in model_info by @simonp9 in #49160
build(deps): bump pyasn1, pypdf and minimatch to clear osv-scan by @dependabot[bot] in #49362
fix(proxy): forward a valid metadata.user_id to Anthropic Messages on pass-through routes by @mika-lt in #49333
fix(openai): stop stripping cache_control from the caller's own messages and tools by @vineethsaivs in #35602
feat!: add Hostinger Router as an OpenAI-compatible provider by @algirdasci in #44863
perf(logging): skip the base64 regex on plain text and copy a few large strings inline by @zoveki in #49228
fix(decisions)!: honor a deployment's own supports_audio_input and supports_video_input flags by @mika-lt in #49207
fix(proxy): trace model list requests with the caller's traceparent by @wileykestner in #49141
fix(proxy): count native Responses WebSocket turn tokens toward key and tag TPM limits by @quietjules in #49257
ci(circleci): clone the public repo over HTTPS so main jobs stop failing at checkout by @zoveki in #49394
fix(anthropic): map pause_turn and return refusal explanations on chat and responses by @paveln83 in #49192
fix(guardrails)!: keep the upstream error when a /v1/messages stream ends in one by @paveln83 in #49088
fix(gemini): add thought_signature_in_tool_call_id to keep tool call ids short for clients that cap them by @mateo-berri in #47692
feat(accounting): capture request usage through the durable meter by @joshua-berri in #49133
fix(proxy): price OpenAI pass-through /v1/decisions calls like native Decisions by @ryan-crabbe-berri in #49392
chore(cost-map): add openai gpt-6-astra-law from the pricing page by @devin-ai-integration[bot] in #49444
fix(cost): split a provider-reported total between input and output by the cost map rate ratio by @kerry-berri in #49357
docs(model-catalog): explain meter, band, channel and When by @yujonglee-berri in #49447
fix(cost): keep upstream_usage_cost on the Responses stream logging copy so bridged /v1/messages rows price their tokens by @kerry-berri in #49365
fix(messages): price a native /v1/messages body as served unless its provider reports its own cost by @kerry-berri in #49356
test(integration): size restart-mid-burst timeouts from the proxy boot budget by @yucheng-berri in #49376
fix(streaming): normalize stop after OpenAI-compatible tool calls by @tin-berri in #49388
fix(router): inspect image content without translating routed requests by @joshua-berri in #49371
fix(oci): preserve Gemini system messages (#48813) by @ishaan-j-berri in #49458
test(cost): expect cohere rerank-v4 prompt_tokens from billed_units.total_tokens by @kerry-berri in #49455
fix(streaming)!: fail a stream that ends without a finish reason by @yucheng-berri in #47665
fix(scheduler): read shared queue from Redis so expired dead-replica entries stop blocking waiters by @mateo-berri in #49387
fix(watsonx): read pass-through credentials from use_in_pass_through deployments by @kerry-berri in #49462
fix(a2a): drop wrong 10 minutes comment on DEFAULT_A2A_AGENT_TIMEOUT by @moe-berri in #49465
fix(proxy): preserve discovery after budget exhaustion (#48809) by @ishaan-j-berri in #49463
fix(moderations): stop the router image check from 500ing on a list input by @kerry-berri in #49454
fix(a2a): send agent litellm_params.api_key as bearer auth on the A2A hop by @moe-berri in #49474
test(integration): wait for the stalled-tenant root span and accept a bare POST name by @yucheng-berri in #49406
fix(caching): stop counting an unrun server tool call mark, and follow #49192 and #35602 in their integration tests by @mateo-berri in #49468
fix(logging): drop a first-token time left over from a previous attempt by @Bytechoreographer in #47541
docs(pr-template): add a handwritten 1-3 sentence description section by @mateo-berri in #49475
ci: cut the weekly rc branch on Monday 3am PT instead of Friday by @yuneng-berri in #49472
feat(passthrough): add Deepgram streaming text-to-speech WebSocket route by @shrey-berri in #49282
fix(a2a): answer proxy refusals on /a2a POST routes as JSON-RPC errors by @moe-berri in #49476
refactor(router): extract PythonCore behind RouterCore and pass the router its config by @narmstrong-berri in #49119
fix(anthropic): answer 400 naming a non-string role or tool call id instead of 500 by @simonp9 in #49237
fix(cohere): emit streamed tool calls from v2 tool-call-start and tool-call-delta chunks by @pcbeingused333 in #47794
feat(chat): resume replies after browser disconnects by @moe-berri in #49461
docs(pr-template): wrap the agent-drafted summary in make-me-human-readable tags by @mateo-berri in #49494
refactor(ui): remove dead API calls from networking.tsx by @yuneng-berri in #49507
test: deflake the json accumulator drain test and the armed routing prefetch test by @kerry-berri in #49505
fix(proxy): treat empty agent and skill search embedding model as not configured by @moe-berri in #49485
fix(passthrough): fit the Deepgram listen unpriced refusal in a WebSocket close frame by @shrey-berri in #49502
fix(a2a): log authenticated pre-dispatch agent rejections by @moe-berri in #49492
fix(router): restore four base behaviors the RoutingContext refactor changed by @narmstrong-berri in #49395
test(translation): expect the EOF error frame when an Anthropic json_schema stream is cut by @mateo-berri in #49521
fix(proxy): serve /a2a/{agent_id}/message/stream routes by @moe-berri in #49471
fix(http): diagnose aiohttp connection pool saturation by @tin-berri in #49499
fix(streaming): preserve served models in OpenRouter streams by @tin-berri in #49498
feat(chat): add deep research with persistent citations by @moe-berri in #49508
fix(agents): show non-admin callers the proxy URL in agent cards by @moe-berri in #49483
feat(guardrails): return native findings on request (#49217) by @ishaan-j-berri in #49460
feat(sdk): invoke configured guardrails (#49217) by @ishaan-j-berri in #49469
feat(bedrock): configure full guardrail assessments (#49217) by @ishaan-j-berri in #49470
fix(docs): replace stale docs/proxy/enterprise links with the current docs pages by @kerry-berri in #49491
test(router): trace the deployment index lock in the file that owns it by @mateo-berri in #49520
fix(chat): describe general chat and deep research on the chat home by @moe-berri in #49531
test(mcp): cancel disconnected HTTP tool handlers by @mateo-berri in #49436
feat(ovalix)!: scan files, tool calls and tool results by @yucheng-berri in #49309
chore(e2e): remove the unused arize_phoenix callback from the stage mirror config by @yuneng-berri in #49538
fix(cost-map): add 2026-11-04 deprecation_date to together_ai Qwen3.8-2.4T-A95B by @devin-ai-integration[bot] in #49542
fix(bedrock): raise GovCloud West Llama 3 max output to the model card 8K by @devin-ai-integration[bot] in #49543
fix(proxy)!: support reload with multiple workers by @devin-ai-integration[bot] in #45909
test: move Sail, Weave and Presidio e2e tests to integration tests by @yuneng-berri in #49533
test(a2a): replace the third-party agent e2e test with an integration test by @yuneng-berri in #49541
fix(wandb): set DeepSeek-V3.1 and Llama-3.1-8B max_input_tokens from the CoreWeave Forge page by @devin-ai-integration[bot] in #49616
test(integration): backport release qualification repairs to rc 1.109.0 by @yuneng-berri in #49694
test(integration): backport remaining qualification checks to rc/1.109.0 by @yuneng-berri in #49716
test(integration): defer empty-count Gemini cases on rc/1.109.0 by @yuneng-berri in #49746
fix: backport metadata and metric repairs to rc/1.109.0 by @yuneng-berri in #49727
test(integration): finish rc/1.109.0 qualification repairs by @yuneng-berri in #49754
test(integration): fix rc109 ledger transaction expiry by @yuneng-berri in #49770
fix(bedrock): backport #49692 to rc/1.109.0 by @mateo-berri in #49724
test(e2e): backport qualification repairs to rc/1.109.0 by @yuneng-berri in #49945
chore: rebuild Admin UI bundle on rc/1.109.0 by @yuneng-berri in #49978

New Contributors

@bap1106 made their first contribution in #39324
@Gopal3746 made their first contribution in #45715
@DewellNaa made their first contribution in #48335
@dkrisman made their first contribution in #40967
@discobot made their first contribution in #30277
@sidhunt made their first contribution in #43290
@alvcol made their first contribution in #46861
@selimfedakar made their first contribution in #48845
@MaximeBOUDIER made their first contribution in #46902
@algirdasci made their first contribution in #44863

Full Changelog: v1.109.0-dev.2...v1.109.0-rc.1