BerriAI/litellm v1.102.0-rc.1

on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.102.0-rc.1

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.102.0-rc.1/cosign.pub \
  ghcr.io/berriai/litellm:v1.102.0-rc.1

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

• fix(caching): keep an open Redis circuit breaker open and quiet on the sync read and spend counter paths by @devin-ai-integration[bot] in #40624
• fix(mcp): accept VS Code OAuth registration callbacks by @joshua-berri in #40664
• fix(mcp): check OpenAPI specifications without native MCP handshakes by @joshua-berri in #40665
• fix(model_prices): registry audit rolling PR: deepseek-flash, gpt-live-1, xAI/Groq deprecation dates, Perplexity Nemotron reasoning by @devin-ai-integration[bot] in #40606
• perf(proxy): lazy-load provider passthrough routes by @devin-ai-integration[bot] in #40691
• fix(proxy): give user-key objects their own in-memory cache partition by @devin-ai-integration[bot] in #40713
• fix(passthrough): parse Bedrock stream spend incrementally instead of buffering the whole response by @devin-ai-integration[bot] in #40724
• feat(secret_managers): support customer-managed KMS key for virtual keys stored in AWS Secrets Manager by @devin-ai-integration[bot] in #40475
• feat(proxy): make the in-memory management cache capacity configurable by @devin-ai-integration[bot] in #40725
• perf(proxy): register liveness and core inference routes first by @devin-ai-integration[bot] in #40687
• docs(router): name both affinity TTL knobs in the _claim_pin docstring by @anmolg1997 in #40663
• fix(mcp): explain refused OAuth registration and bound discovery retries by @joshua-berri in #40679
• fix(mcp): log upstream request method, body and response on tool-list and OAuth2 token failures by @devin-ai-integration[bot] in #40440
• fix(oci): pin one response id per streamed completion, skip the [DONE] sentinel by @mateo-berri in #39507
• fix(vector-stores): surface retrieval failures to the API caller by @mateo-berri in #39516
• fix(ai-gateway): build the release image again and cover it in CI by @mateo-berri in #39523
• fix(proxy): authorize every Responses API id, not only the ones the proxy issued by @mateo-berri in #39548
• fix(router): fall back from unhealthy auto-router tier by @devin-ai-integration[bot] in #40757
• fix(proxy): keep call_type and request start time on failed-request spend logs by @devin-ai-integration[bot] in #40558
• fix(proxy): retain metadata when retrieving public team aliases by @haydster7 in #40554
• test(e2e): wait for serving propagation in UI journeys by @yuneng-berri in #40754
• refactor(ocr): route native requests through core by @yujonglee-berri in #40532
• feat(auto-router): show the routed model and session savings in Claude Code and Codex by @tin-berri in #40330
• fix(mcp): write failure spend log for guardrail-blocked /mcp-rest/tools/call by @devin-ai-integration[bot] in #40555
• feat(ocr): add Azure Mistral adapter and document fetching by @yujonglee-berri in #40533
• fix(datadog_llm_obs): keep tool call and result structure under redaction and emit tool output tokens by @devin-ai-integration[bot] in #40666
• fix(search): propagate GET provider HTTP errors by @yuneng-berri in #40779
• fix(policy_engine): run global policy pipelines before scoped ones by @devin-ai-integration[bot] in #39697
• fix(content_filter): log only scan time as streaming post_call guardrail duration by @devin-ai-integration[bot] in #40760
• fix(cost-map): bedrock reasoning effort flags, registry audit fixes for vertex/openai/together/openrouter, absorb cerebras and inception rows by @devin-ai-integration[bot] in #40740
• test(responses): fix stale Anthropic smoke request by @yuneng-berri in #40786
• fix(ui): jump straight to the last Request Logs page instead of advancing one page by @devin-ai-integration[bot] in #40644
• feat(ocr): add Azure Mistral adapter with native authentication by @yujonglee-berri in #40502
• feat(ocr): add Azure Document Intelligence adapter by @yujonglee-berri in #40534
• feat(ocr): add Reducto legacy and v3 adapters by @yujonglee-berri in #40535
• feat(ocr): add Vertex Mistral adapter by @yujonglee-berri in #40507
• feat(ocr): add Vertex DeepSeek adapter and remove legacy OCR pipeline by @yujonglee-berri in #40509
• fix(vertex_ai): return 400 for invalid reasoning_effort instead of 500 by @ryan-crabbe-berri in #40748
• fix(guardrails): stop logging the request payload as guardrail_response on pre_call hooks by @devin-ai-integration[bot] in #39699
• feat(rust): count tiktoken cl100k_base admission tokens in Rust by @devin-ai-integration[bot] in #40777
• fix(hide-secrets): restore credential coverage lost to the 4.5 entropy limit by @yucheng-berri in #40190
• test(load): add a Redis timeout chaos load test by @kerry-berri in #40482
• feat(proxy): expose complexity routing headers by @devin-ai-integration[bot] in #40792
• fix(otel v2): name Langfuse traces from the langfuse_trace_name header or metadata.trace_name by @devin-ai-integration[bot] in #40793
• feat(ui): link the Organization cell on the Teams page by @ryan-crabbe-berri in #40749
• docs(pr-template): note untested assumptions under Caveats by @kerry-berri in #40811
• feat(ui): link the User ID and Team ID cells on the Memory page by @ryan-crabbe-berri in #40752
• feat(ui): link the User ID, Created By and Deleted By cells on Deleted Keys by @ryan-crabbe-berri in #40750
• feat(ui): link the Created By cell on the Prompts page by @ryan-crabbe-berri in #40753
• feat(ui): link the Organization and Deleted By cells on Deleted Teams by @ryan-crabbe-berri in #40751
• fix(proxy): emit internal user budget webhook alerts by @jon-walton in #40396
• fix(proxy): prevent spend counter double counting by @dclarksymmetry in #40572
• fix(guardrails): keep post_call guardrail info on streamed chat completions by @devin-ai-integration[bot] in #40806
• fix(realtime): dial Azure's GA realtime upstream for GA clients by @devin-ai-integration[bot] in #40769
• fix(ui): preserve cleared shared select values by @yuneng-berri in #40795
• feat(model_prices): add DeepSeek V4.1 Flash on Fireworks by @devin-ai-integration[bot] in #40812
• fix(fireworks_ai): keep reasoning_content on replayed assistant messages by @devin-ai-integration[bot] in #40682
• fix(shadow-eval): skip hosted web search samples by @devin-ai-integration[bot] in #40827
• fix(db): carry DATABASE_SSLMODE/DATABASE_SSLROOTCERT into the assembled writer and reader URLs by @devin-ai-integration[bot] in #40815
• fix(guardrails): fail closed with a named error when a Responses input rewrite cannot be applied by @csoni-cweave in #40609
• fix(ui): make the env-credential login warning banner dismissible by @devin-ai-integration[bot] in #40831
• test(pgbouncer): stop the never-listens replacement test flaking under CI load by @devin-ai-integration[bot] in #40830
• feat(ui): search, sort and role filter for the team member table by @devin-ai-integration[bot] in #40659
• fix(proxy): keep the raw model string out of the unknown-model spend-log error message by @devin-ai-integration[bot] in #40820
• fix(mcp): cache upstream discovery lists by @joshua-berri in #40790
• fix(mcp): match per-server OAuth metadata issuers by @joshua-berri in #40808
• fix(mcp): use gateway authentication for root discovery by @joshua-berri in #40791
• fix(bedrock_mantle): gate reasoning.summary on the OpenAI Responses path by @devin-ai-integration[bot] in #40798
• test(e2e): memory regression test for failing requests on the release gate by @devin-ai-integration[bot] in #40773
• test(e2e): reusable JWT fixtures and management lifecycle coverage by @ryan-crabbe-berri in #39990
• fix(proxy): expand access groups in /health scoping and allowlist health display fields by @devin-ai-integration[bot] in #40765
• test(ocr): exempt native parity requests from cassette replay by @yuneng-berri in #40838
• fix(proxy): persist clearing user model budgets by @yuneng-berri in #40837
• perf(auth): read user, team, membership, org, project and spend counters in one MGET, one query and one pipeline by @devin-ai-integration[bot] in #40834
• perf(proxy): one MGET and one pipeline for post-call spend counters, no team/user/org refetch on the response path by @devin-ai-integration[bot] in #40841
• fix(responses): preserve hosted web search calls by @devin-ai-integration[bot] in #40828
• feat(cli): configure Claude Code and Codex with a gateway key by @tin-berri in #40829
• fix(router): restore compression inheritance when clearing overrides by @yuneng-berri in #40839
• fix(keys): support explicit project detachment by @yuneng-berri in #40836
• feat(guardrails): add Conduct Guard integration with validated hooks and forwarded params by @devin-ai-integration[bot] in #40785
• fix(mcp): enforce end user mcp_tool_permissions on tools/list and tools/call by @devin-ai-integration[bot] in #40865
• refactor(ocr): complete native lifecycle and preserve Azure auth by @yujonglee-berri in #40734
• fix(ui): restore MCP catalog provider logos by @yuneng-berri in #40781
• fix(ui): preserve clear and default semantics in local forms by @yuneng-berri in #40826
• fix(ui): clarify blank TPM/RPM hint on budget modals by @devin-ai-integration[bot] in #40697
• test(e2e): verify cached answers and upstream request count by @yuneng-berri in #40774
• fix(key): recover from a cascade-deleted key instead of failing the apply by @ryan-crabbe-berri in #40880
• fix(proxy): accept both deferred stream logging arg shapes on native routes by @devin-ai-integration[bot] in #40869
• perf(policy_engine): dedup attachments in one pass after sorting by @devin-ai-integration[bot] in #40883
• fix(ui): persist cleared budgets and reset intervals by @yuneng-berri in #40895
• fix(guardrails): log mask when a guardrail adds request keys by @devin-ai-integration[bot] in #40882
• fix(ui): show loading state instead of stale rows while a table search is pending by @devin-ai-integration[bot] in #40656
• chore: bump litellm-enterprise 0.1.66 -> 0.1.67, litellm-proxy-extras 0.4.96 -> 0.4.97 by @yuneng-berri in #40905
• test(fireworks): stop pinning prices in the cost-map tests by @devin-ai-integration[bot] in #40901
• feat(ui): search Key Activity by key alias, key hash, user id, or email by @devin-ai-integration[bot] in #40652
• fix(proxy): run migrations through python -m prisma when the prisma console script is not on PATH by @devin-ai-integration[bot] in #40768
• feat(proxy): serve registered skills as an Agent Skills well-known index by @devin-ai-integration[bot] in #40770
• test: deflake redis semantic cache sys.modules leak, LangSmith init loop patch, wall-clock stagger assertion, and zombie grandchild check in the fake prisma cli by @devin-ai-integration[bot] in #39895
• fix(anthropic): price recovered tokens when a /v1/messages client disconnects mid-stream by @devin-ai-integration[bot] in #40766
• fix(proxy): resolve config include directives for bucket-hosted configs by @devin-ai-integration[bot] in #40772
• fix(registry): sync Azure/Together deprecation dates, fix computer-use-preview and OpenRouter metadata by @devin-ai-integration[bot] in #40855
• test: tighten regression tests added in #37974 by @devin-ai-integration[bot] in #40771
• fix(redis): count pool wait timeouts as breaker timeouts by @devin-ai-integration[bot] in #40764
• fix(cost): honour deployment custom pricing for OCR calls by @devin-ai-integration[bot] in #40767
• fix(cost): honour deployment custom pricing for OCR calls by @mihidumh in #36609
• feat(realtime): add Meta Muse Voice transcription by @seyeong-han in #39395
• fix(proxy): attribute gate-rejected requests to their endpoint in cache analytics by @devin-ai-integration[bot] in #40824
• feat(registry): add openai reasoning-family fallback generalization by @devin-ai-integration[bot] in #40902
• fix(proxy): allow key_alias substring matching on /key/list for non-admins by @devin-ai-integration[bot] in #40907
• fix(databricks): translate reasoning_effort to thinking for Gemini 2.5 by @devin-ai-integration[bot] in #40909
• fix(databricks): translate reasoning_effort to thinking for Gemini 2.5 by @cohml in #32786
• fix(model_management): stop persisting cost map pricing as a deployment override by @ryan-crabbe-berri in #36222
• chore: rebuild Admin UI bundle from staging by @yuneng-berri in #40931
• chore(ci): promote internal staging to main by @yuneng-berri in #40933

New Contributors

• @haydster7 made their first contribution in #40554
• @kerry-berri made their first contribution in #40482
• @jon-walton made their first contribution in #40396
• @dclarksymmetry made their first contribution in #40572

Full Changelog: v1.102.0-dev.2...v1.102.0-rc.1