BerriAI/litellm v1.101.0-dev.2 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.101.0-dev.2

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.101.0-dev.2/cosign.pub \
  ghcr.io/berriai/litellm:v1.101.0-dev.2

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

test: exempt MockTransport request-shape embedding tests from VCR replay by @mateo-berri in #39185
fix(ui): render the logs Tools panel with theme tokens by @yuneng-berri in #39129
fix(proxy): default max_idle_connection_lifetime to 60s on DB URLs by @devin-ai-integration[bot] in #39134
fix(mcp): follow tools/list pagination from upstream servers by @yucheng-berri in #39172
fix(proxy): resolve router model aliases in /utils/supported_openai_params by @mateo-berri in #39000
fix(azure): flatten top-level tool schema combinators on Azure chat completions by @mateo-berri in #38870
fix(bedrock): route streamed responses-API output through the unified guardrail by @mateo-berri in #38734
fix(ui): hide model write affordances from view-only admin sessions by @mateo-berri in #38872
fix(cli): quote the Claude Code apiKeyHelper for cmd.exe on Windows by @devin-ai-integration[bot] in #39174
fix(logging): guarantee max_parallel_requests slot release when streaming logging fails by @devin-ai-integration[bot] in #39093
feat(alerting): slack alerts for per-user daily/monthly spend thresholds and spend anomaly detection by @devin-ai-integration[bot] in #38438
fix(docker): add public Wolfi apk repo to runtime image by @devin-ai-integration[bot] in #39033
fix(router): keep order fallback on the requested order level by @emerzon in #38969
test(e2e): cover retry-on-timeout and the context-window fallback by @ryan-crabbe-berri in #39197
fix(budget): reject known estimates over remaining budget under fail_closed_budget_enforcement by @devin-ai-integration[bot] in #39214
fix: stop a cleared Team field from blocking personal key creation by @ryan-crabbe-berri in #39206
test: record each e2e test's source location in the JUnit report by @ryan-crabbe-berri in #39209
feat(router): fall back on anthropic safeguard refusals on /v1/messages by @tin-berri in #39157
fix(proxy): report requested model on Anthropic streaming message_start by @devin-ai-integration[bot] in #35816
fix(helm): reuse the generated master key Secret on helm upgrade by @devin-ai-integration[bot] in #39219
fix(mcp): report per-server outcomes in aggregate REST tools/list by @devin-ai-integration[bot] in #39232
fix(cost-map): retry transient boot fetch failures and recover config deployments dropped by a stale cost map by @devin-ai-integration[bot] in #39230
perf(scim): resolve group members with one user table read per member by @devin-ai-integration[bot] in #39228
fix(docker): install bedrock-realtime extra in monolith proxy images by @devin-ai-integration[bot] in #39223
fix(aiohttp_transport): map transport-internal CancelledError to a retryable ConnectError by @devin-ai-integration[bot] in #39240
fix(bedrock): gate Converse cachePoint emission on model prompt caching support by @tin-berri in #39210
fix(datadog_llm_obs): send tool calls, tool results and cache tokens in DD's own fields by @tin-berri in #39222
feat(prometheus): expose per-key and per-team rate limit allowed and used gauges by @devin-ai-integration[bot] in #39236
feat(scim): add placeholder listing and merge so a shadowed account can be healed by @devin-ai-integration[bot] in #39231
fix: normalize provider-specific cache token fields in OTel v2 usage by @yucheng-berri in #39202
fix: stop deployment default API key limits leaking into provider requests by @devin-ai-integration[bot] in #39211
fix(proxy): keep passthrough logging metadata and model_info dicts when team callbacks are wired by @yucheng-berri in #39216
fix(guardrails): deliver modify_response block as valid SSE on streaming chat and Responses by @mateo-berri in #39036
fix(search): forward search-tool params through the router, complete Parallel AI v1 param mapping by @jliounis in #37883
fix(bedrock): stop Converse crashing on bearer-token auth without SigV4 credentials by @mateo-berri in #39166
fix(docker): install saml extra in litellm-backend image by @ojensen-berri in #39291
fix(guardrails): run apply_guardrail-only providers in logging_only mode by @devin-ai-integration[bot] in #39297
feat(gemini): day-0 pricing for gemini-3.8-flash by @mateo-berri in #39340
fix(vertex): avoid duplicate DeepSeek OCR model namespace by @yujonglee-berri in #39194
feat(streaming): carry final response cost on streamed usage by default by @mateo-berri in #39069
fix(rerank): map provider errors with the resolved provider on sync and async paths by @mateo-berri in #39176
test(e2e): read JUnit properties off the real collected pytest Item by @mateo-berri in #39246
feat(proxy): configurable display_name for the Anthropic-shaped /v1/models listing by @mateo-berri in #39238
fix(helm): scale the classic chart's HPA out at the documented 60 percent CPU by @yassin-berriai in #35975
fix(gemini): return enabled thinking content by default by @mateo-berri in #39160
fix: run access group key sync UPDATEs on the writer, not the read replica by @devin-ai-integration[bot] in #39128
fix(models): registry audit 2026-09-01: openai realtime and long-context tiers, mistral aliases, voyage, xai, fireworks, together, scaleway, azure ai, govcloud, azure gov, cloudflare whisper, deprecation dates by @devin-ai-integration[bot] in #39170
fix: apply optional_pre_call_checks and reject unsupported router settings on /config/update by @devin-ai-integration[bot] in #39249
fix(vector_stores): s3 vectors search router bypass + rag query config drop + ui error swallow by @michelligabriele in #34788
fix(models): key Azure DeepSeek V4 Flash 0731 by its Foundry catalog id by @yujonglee-berri in #39341
fix(deps): raise the tornado and pypdf floors for six new advisories by @mateo-berri in #39188
fix(headroom): stop re-compressing retrieved CCR content in client tool loops by @QuantumBreakz in #38591
feat(agentcore-a2a): derive runtime session id from A2A message.contextId by @devin-ai-integration[bot] in #39371
fix(proxy): share per-model budget counters across replicas through the spend counter cache by @devin-ai-integration[bot] in #39375
fix(proxy-extras): give prisma migrate deploy its own timeout budget by @mateo-berri in #39365
fix(proxy): route container create and list through model_list deployments by @devin-ai-integration[bot] in #39220
test(build): validate release wheel contracts by @yujonglee-berri in #39021
refactor(rust): extract domain-neutral Python interop by @yujonglee-berri in #39026
refactor(rust): standardize the core Error type by @yujonglee-berri in #39331
fix(ui): preserve full AgentCore runtime ARN in agent edit form by @yassin-berriai in #39382
feat(ui): update OpenAI preset model tiers by @tin-berri in #39396
fix(router): resolve realtime session model to routed deployment by @devin-ai-integration[bot] in #36811
fix(security): restrict and validate file uploads at /v1/files and /upload/logo by @yassin-berriai in #39379
feat(auth): enforce configurable password policy and SSO-only login by @yassin-berriai in #39381
fix(agents): redact secret litellm_params fields from all /v1/agents responses by @yassin-berriai in #39389
fix(otel): stamp Langfuse root observation input and output from the request task by @mateo-berri in #39369
fix(guardrails): track and tear down presidio sibling callbacks on delete and update by @mateo-berri in #39271
fix(spend): keep every-deployment scope on gateway cache-injection marks by @mateo-berri in #39241
fix(proxy/db): keep prisma predicates from raising TypeError under a mocked prisma module by @devin-ai-integration[bot] in #39253
fix(proxy): word database 503s by whether the fault is transient by @devin-ai-integration[bot] in #39256
refactor(utils): remove the dead get_api_key provider-key resolver by @devin-ai-integration[bot] in #39260
feat(mcp): semantic tool search for the native MCP Gateway by @devin-ai-integration[bot] in #39404
fix(logging): redact credential query params from the uvicorn access log by @yucheng-berri in #39293
feat(model_prices): add meta/muse-spark-1.3 and its contributor tier by @devin-ai-integration[bot] in #39417
refactor(core): move audio transcription into core by @yujonglee-berri in #39126
fix(proxy): build coordination Redis from REDIS_* env vars unconditionally by @yassin-berriai in #39410
test: add interactive Rust Python parity harness by @ishaan-berri in #39419
test(proxy): verify NO_DOCS/NO_REDOC/NO_OPENAPI restrict every doc surface by @yassin-berriai in #39378
test(bedrock): accept the router kwarg in the knowledge base search fake by @mateo-berri in #39420
refactor(python-bridge): split routes and add shared function tracing by @yujonglee-berri in #39031
fix(python-bridge): harden sync and async execution boundaries by @yujonglee-berri in #39332
refactor(python-bridge): declare sync and async routes once by @yujonglee-berri in #39333
feat(python): unify Rust opt-in and bridge policy by @yujonglee-berri in #39334
feat(router): add heuristic v2 complexity routing by @tin-berri in #39276
fix(anthropic): upgrade legacy thinking to adaptive on adaptive-only Claude models for chat, Bedrock Converse, Invoke, Vertex AI, and Databricks by @devin-ai-integration[bot] in #39159
fix(proxy): mark session/SSO/SAML cookies Secure behind a TLS-terminating reverse proxy by @yassin-berriai in #39391
fix(bedrock): honor BEDROCK_MANTLE_API_BASE on bedrock/mantle messages and chat URLs by @mateo-berri in #39364
fix(bedrock): strip client_metadata from converse additionalModelRequestFields by @devin-ai-integration[bot] in #35967
chore(techdebt): clear fresh debt from the 2026-08-31 and 2026-09-01 windows by @devin-ai-integration[bot] in #39091
fix(mcp): cap tools preview and test-connection at the listing timeout and name the unreachable upstream by @mateo-berri in #38791
fix(hosted_vllm): forward truncate_prompt_tokens on rerank requests by @mateo-berri in #39363
fix(messages): drop cache_control ttl on non-Anthropic /v1/messages passthrough by @mateo-berri in #39355
fix(bedrock_mantle): carry per-request AWS credentials into chat completions SigV4 signing by @mateo-berri in #39362
feat(router): add a hybrid classifier that defers near tier boundaries by @tin-berri in #39403
fix: recover the v2 migration resolver from concurrent migrate deploy deadlocks by @mateo-berri in #39187
fix(ollama_chat): stamp finish_reason tool_calls when tool calls streamed before the done chunk by @devin-ai-integration[bot] in #39010
fix(router): route Claude Code subagents through session router by @moe-berri in #39239
fix(responses): keep namespace tools intact when a guardrail returns them unchanged by @mateo-berri in #39366
fix(vector-store): resolve embedding credentials per request by @yujonglee-berri in #38936
test(e2e/ui): give the seeded users passwords that pass the default password policy by @mateo-berri in #39442
fix(http_handler): honor HTTP(S)_PROXY / NO_PROXY when force_ipv4 uses the httpx transport by @devin-ai-integration[bot] in #39443
fix(proxy): stop leaking internal exception details to clients by @yassin-berriai in #39380
fix(guardrails): forward mode and streaming params to crowdstrike_aidr handler by @devin-ai-integration[bot] in #39317
fix(mcp): gate the connect-time OBO pre-flight on the key's allowed servers by @devin-ai-integration[bot] in #39447
fix(responses): keep provider response headers in streaming logging callbacks by @devin-ai-integration[bot] in #38131
fix(mcp): fence an outbound-token write against an overlapping invalidation by @yassin-berriai in #35398
feat(cli): pre-fill the SSO verification code in the browser when the proxy allows it by @mateo-berri in #39428
fix(ui): paginate request logs by session groups server-side by @tin-berri in #39257
feat(proxy): serve the auto-router preset catalog at runtime by @tin-berri in #39412
docs: define Rust Python harness structure by @yujonglee-berri in #39456
fix(guardrails): apply PUT /guardrails/{id} to the serving worker immediately and reject invalid configs with 422 by @mateo-berri in #38877
test(responses): expect the 404 OpenAI now returns for an unknown model by @mateo-berri in #39457
fix(guardrails): skip streaming guardrail rounds that re-scan cleared output by @mateo-berri in #39386
fix: keep litellm importable on Python 3.10 and guard 3.11-only typing imports in CI by @devin-ai-integration[bot] in #39448
fix(proxy): keep SpendLogs and callback session ids in sync when the request has none by @devin-ai-integration[bot] in #39450
feat(router): arm safeguard-refusal fallback on generic chains when no content-policy list exists by @tin-berri in #39274
feat(azure): support credential chain for storage by @yucheng-berri in #39229
chore(crowdstrike): expect the deduped end-of-stream scan in crowdstrike cadence test by @devin-ai-integration[bot] in #39467
fix(model_armor): handle Anthropic Messages and Responses streams in post_call by @yucheng-berri in #39181
test: add OCR python-to-rust test parity ledger (WIP) by @ishaan-berri in #39434
feat(complexity_router): opt-in modality override of a kept session-affinity pin by @tin-berri in #39454
feat(datadog_llm_obs): cost tag dimensions, router decision fields, reasoning token metric, redaction gating by @yucheng-berri in #39402
test(rust-python-harness): wire existing e2e SDK tests into the matrix by @ishaan-berri in #39463
fix(mcp): never exchange the LiteLLM virtual key as the upstream subject token by @devin-ai-integration[bot] in #39446
test: add mistral ocr transformation parity coverage by @ishaan-berri in #39482
test(vector-store): accept embedding_executor in the Bedrock KB hook fake handler by @mateo-berri in #39472
refactor(s3_vectors): embed search queries through the shared vector store executor by @mateo-berri in #39474
fix(xai): bill from the cost xAI reports instead of recomputing it (internal copy of #36281) by @mateo-berri in #39441
feat(ui): add 1M context auto-router preset by @tin-berri in #39490
fix(ui): stop the create team form resetting organization and models by @yuneng-berri in #39476
fix(ui): read the preset catalog at runtime in the dashboard tests by @tin-berri in #39478
fix(sso): resolve multi-valued role claims to the highest privilege role by @yuneng-berri in #39480
fix(guardrail): hide-secrets playground redaction and guardrail telemetry by @yucheng-berri in #39398
fix(test): drop the duplicate embedding_executor arg in the Bedrock KB fake handler by @yuneng-berri in #39502
fix(ui): keep Virtual Keys list state in the URL so it survives leaving the page by @yuneng-berri in #39481
fix(proxy): 404 a credential delete that matched nothing, and raise instead of return by @eeshsaxena in #36260
fix(proxy-extras): only spend a migrate-deploy attempt when a pass made no progress by @mateo-berri in #39506
feat(cli): enable Claude Code gateway model discovery by default in lite claude by @devin-ai-integration[bot] in #39445
fix(docker): bump nginx runtime to 1.31.5-alpine3.24 and pin digest by @rakeshrepository in #39561
fix: 1.99.0-rc2 UI bug batch (empty org on key create, session pagination, access group rename/delete) by @mateo-berri in #39436
feat(auto-router): support classifier reasoning effort by @moe-berri in #39372
fix(ui): replace the key detail URL entry when a virtual key is rotated by @yuneng-berri in #39471
test(timeout): time out against the local fake endpoint instead of api.openai.com by @yuneng-berri in #39583
test(harness): add OCR parity with migration strategy runners by @yujonglee-berri in #38765
fix(databricks): strip thinking_blocks and reasoning_content from outbound messages by @devin-ai-integration[bot] in #39409
test(ocr): record provider fixtures in the migration harness by @yujonglee-berri in #39425
feat(ui): keyset-paginate request logs by session trace by @devin-ai-integration[bot] in #38794
fix(proxy/db): translate libpq sslrootcert and verify-* into Prisma's strict TLS params by @devin-ai-integration[bot] in #39563
fix(agents): keep the published agent in public_agent_groups by @mateo-berri in #39554
fix(mcp): scope allow-all servers to virtual keys by @tin-berri in #39531
fix(team): generate team IDs for blank input by @yujonglee-berri in #39571
fix(bedrock_mantle): stop dropping the web_search tool on /v1/responses by @devin-ai-integration[bot] in #35987
chore: bump litellm-enterprise 0.1.63 -> 0.1.64, litellm-proxy-extras 0.4.92 -> 0.4.93 by @yuneng-berri in #39595
fix(images): forward gpt-image supported params like background to OpenAI and Azure by @devin-ai-integration[bot] in #39525
fix(proxy): return persisted team memberships from /user/new so first CLI login gets the default team by @devin-ai-integration[bot] in #39545
fix(spend_tracking): add missing_session_id: omit to leave SpendLogs.session_id null without a client session by @devin-ai-integration[bot] in #39458
fix: stop a cleared Organization field from failing key creation by @devin-ai-integration[bot] in #39316
fix(ui): show MCP servers and agents inherited from access groups on team overview by @ryan-crabbe-berri in #39215
fix(proxy): expose configured mode for auto-router models by @moe-berri in #39619
fix(ui): aggregate session token usage in the logs table by @ryan-crabbe-berri in #39598
fix(cost): apply off_peak_pricing in the dashscope cost calculator by @mateo-berri in #39592
test(bedrock): drop EOL cohere.command-r-plus-v1:0 from local_testing by @mateo-berri in #39608
fix(openai): default stream usage on PrivateLink and regional api.openai.com hosts by @mateo-berri in #39614
fix(proxy): drop anthropic-beta on the Vertex passthrough count-tokens route by @mateo-berri in #39597
fix(headroom): resolve CCR retrieval on streaming /v1/responses by @mateo-berri in #38808
fix(openai): bridge gpt-5.4+ tool calls to /v1/responses on every api.openai.com host by @mateo-berri in #39587
fix(router): pin JWT-authenticated callers by user id in deployment_affinity by @mateo-berri in #39594
fix(cost): bill bedrock_mantle web search at $12 per 1k queries using Bedrock's reported count by @mateo-berri in #39610
fix(azure_ai): don't reclassify Foundry deployments as azure provider by @mateo-berri in #38975
fix(vector_stores): only list vector stores the caller was granted by @devin-ai-integration[bot] in #39612
feat(models): add gpt-6-astra pricing and metadata by @mateo-berri in #39622
chore(ci): promote internal staging to main by @yuneng-berri in #39593
feat(router): limit heuristic_v2 auto-routers to one without the auto_router license feature by @tin-berri in #39468
fix(ui): clear agents when updating team permissions by @devin-ai-integration[bot] in #39600
fix(auto_router): bill the routing embedding to the caller's key and team by @devin-ai-integration[bot] in #39532
test(router): cover get_configured_mode so router_code_coverage passes by @mubashir1osmani in #39630
fix: treat gpt-6 names as the gpt-5 request family in OpenAI and Azure configs by @mateo-berri in #39631
fix(prompts): key the in-memory prompt registry by environment by @mateo-berri in #38440
fix(ui): let the Internal Users search box match user_id as well as email by @ryan-crabbe-berri in #39604
test(responses): bound the background stream cancel e2e so an upstream stall skips fast by @mateo-berri in #39617
fix(vertex): add the API version to versionless project routes on the Vertex passthrough by @mateo-berri in #39625
chore(ci): promote internal staging to main by @yuneng-berri in #39648

New Contributors

@jliounis made their first contribution in #37883
@ojensen-berri made their first contribution in #39291
@QuantumBreakz made their first contribution in #38591
@eeshsaxena made their first contribution in #36260
@rakeshrepository made their first contribution in #39561

Full Changelog: v1.101.0-dev.1...v1.101.0-dev.2