BerriAI/litellm v1.103.0-rc.1 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.103.0-rc.1

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.103.0-rc.1/cosign.pub \
  ghcr.io/berriai/litellm:v1.103.0-rc.1

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

test(cost_map): stop pinning supports_reasoning absent on the openrouter o1 entry by @devin-ai-integration[bot] in #41734
fix(model_prices): add cache-read pricing to Mistral chat models missing it by @devin-ai-integration[bot] in #41736
refactor(rust): isolate legacy callback contract by @devin-ai-integration[bot] in #41752
test(cost_map): drop tests that pin provider-owned catalog values by @devin-ai-integration[bot] in #41763
test(e2e/ui): wait for the filtered budget list before clicking a row action by @yuneng-berri in #41756
chore(prices): sync OpenRouter prices: 443 models, 191 new, 4 deprecated by @berriai-litellm-provider-info-sync[bot] in #41727
fix(batches): bill Bedrock Titan embedding batch lines from inputTextTokenCount by @devin-ai-integration[bot] in #41767
fix(proxy): persist only the keys a caller changed in save_config by @yuneng-berri in #41748
chore(prices): sync OpenRouter prices: 2 models, 1 deprecated by @berriai-litellm-provider-info-sync[bot] in #41770
fix(proxy): run prompt injection heuristics off the event loop by @devin-ai-integration[bot] in #41541
fix(proxy): evict jwt key mapping cache on user, team, org, and bulk key deletion by @devin-ai-integration[bot] in #41707
refactor(ui): remove unused HelpLink and HelpIcon components by @devin-ai-integration[bot] in #41644
chore(ui): remove orphaned ROLE_STYLES and RoleStyle from pretty messages view by @devin-ai-integration[bot] in #41645
chore(ui): remove never-rendered GuardrailConfig mock component and its test by @devin-ai-integration[bot] in #41646
chore(ui): remove unused NewBadge component and its test by @devin-ai-integration[bot] in #41647
chore(ui): remove dead compareUI ModelSelector and its test by @devin-ai-integration[bot] in #41649
refactor(ui): remove unused createCredentialFromModel helper and CredentialValues interface by @devin-ai-integration[bot] in #41650
refactor(ui): drop unused cost-tracking barrel re-exports and response types by @devin-ai-integration[bot] in #41651
refactor(ui): drop unused rolesAllowedToSeeUsage, viewOnlyRoles and isViewOnlyRole exports by @devin-ai-integration[bot] in #41653
chore(ui): remove unused access-groups type interfaces by @devin-ai-integration[bot] in #41655
refactor(ui): remove dead networking exports and orphaned Claude Code marketplace helpers by @devin-ai-integration[bot] in #41656
refactor(ui): remove dead useKeyList hook from key_list.tsx by @devin-ai-integration[bot] in #41657
chore(ui): remove orphaned useDeleteProject hook and its test by @devin-ai-integration[bot] in #41658
refactor(interactions): remove expired use_legacy_interactions_schema shim by @devin-ai-integration[bot] in #41663
refactor(rust): align crates with Python package layering by @devin-ai-integration[bot] in #41829
chore(prices): sync OpenRouter prices: 15 models, 6 deprecated by @berriai-litellm-provider-info-sync[bot] in #41772
refactor(proxy): make the config file win over the database by @yuneng-berri in #41779
fix(ui): let admins change a model's team from the model edit page by @devin-ai-integration[bot] in #40700
fix(ui): show per-second pricing for video models instead of $0.00 token costs by @devin-ai-integration[bot] in #39308
fix(cost): bill cache-read tokens at the input rate when the map has no cache-read rate by @devin-ai-integration[bot] in #41832
fix(responses): honor nested additional_drop_params paths by @devin-ai-integration[bot] in #40730
feat(router): reject with 429 when a deployment's max_parallel_requests slots are all in use by @devin-ai-integration[bot] in #41555
perf: defer fastapi and tiktoken BPE imports out of import litellm by @devin-ai-integration[bot] in #41585
feat(proxy): add Amazon Transcribe pass-through with completion-time job pricing by @devin-ai-integration[bot] in #41515
feat(proxy): per-key default budget for dynamically created customers by @devin-ai-integration[bot] in #41636
feat(mcp): show live gateway sessions by AI client and user by @devin-ai-integration[bot] in #41692
ci: replace the title-similarity duplicate bot with a Codex semantic check by @ryan-crabbe-berri in #40935
feat(proxy): temporary budget increase for team members by @devin-ai-integration[bot] in #41620
feat(vault): add separate login and secret namespaces for HashiCorp Vault by @devin-ai-integration[bot] in #41539
feat(vertex_ai): stream GCS batch output files from /v1/files/{id}/content by @devin-ai-integration[bot] in #41506
feat(a2a): reach Microsoft Foundry agents with Entra auth and versioned card discovery by @devin-ai-integration[bot] in #41511
fix(azure): strip litellm format field from file and image content parts by @devin-ai-integration[bot] in #41275
ci: classify new issues into domain, provider, kind, priority and lift labels by @ryan-crabbe-berri in #41695
fix(router): validate routing_groups at save time and keep invalid DB groups from blocking SSO load by @devin-ai-integration[bot] in #41351
fix(bedrock): send aws_session_tags on every STS call via one typed auth struct by @ryan-crabbe-berri in #40500
fix(anthropic): register thinking-binding-controls-2026-08-01 in beta headers config by @clonylu in #41203
feat(guardrails): add TypeSafe Jev relevance-based compaction guardrail by @devin-ai-integration[bot] in #41757
feat(keys): let team service account keys use key management endpoints for their own team by @devin-ai-integration[bot] in #40807
test(docs): read only the first column of the router_settings reference table by @devin-ai-integration[bot] in #41860
fix(scim): accept entitlements and roles entries without a value on SCIM user PUT by @devin-ai-integration[bot] in #41830
perf(proxy): split aggregated usage query into key-free rollups and bounded top-N keys by @devin-ai-integration[bot] in #41293
chore(deps): bump anyio from 4.13.0 to 4.14.2 by @dependabot[bot] in #41846
feat(proxy): limit repeated failed Admin UI sign-in attempts by @devin-ai-integration[bot] in #40982
fix(proxy): make SettingsStore.clear() terminate when the config file owns a key by @devin-ai-integration[bot] in #41862
feat(vscode): add LiteLLM language model provider extension by @devin-ai-integration[bot] in #41865
feat(mcp): let proxy admins force-close live MCP sessions and revoke stored user credentials by @devin-ai-integration[bot] in #41725
feat(models): add qwen3.8 flash rows, fix Cohere embed v3 context, Bedrock Mantle and OpenRouter pricing by @devin-ai-integration[bot] in #41754
fix(azure): keep api-version query after vector store search path by @devin-ai-integration[bot] in #41384
fix(router): honor stream_timeout on the SDK-native passthrough route (/v1/messages, /converse) by @devin-ai-integration[bot] in #41875
fix(proxy): track team member spend when the member has no budget by @devin-ai-integration[bot] in #41349
feat(proxy): add Azure AI Speech pass-through route by @devin-ai-integration[bot] in #41557
fix(responses): stop managed Responses WebSocket from leaking litellm_params into provider request body by @devin-ai-integration[bot] in #33101
feat(rust): port exception_type to litellm-core-utils by @devin-ai-integration[bot] in #41873
fix(ui): show internal user email in logs table and log detail drawer by @devin-ai-integration[bot] in #40737
fix(router): report null cost for unpriced deployments instead of 0 by @devin-ai-integration[bot] in #40878
feat(azure_ai): support FLUX.2 flex images by @emerzon in #39424
fix(mistral): accept reasoning_effort on all models and drop client_metadata for Codex compatibility by @devin-ai-integration[bot] in #41062
feat(passthrough): deepgram streaming /v1/listen WebSocket passthrough with duration-based cost tracking by @devin-ai-integration[bot] in #41554
test(ocr): declarative provider x auth x input matrix, fix Rust 401 mapping by @devin-ai-integration[bot] in #41884
fix(proxy): classify Azure Speech short audio behind a prefixed api base by @devin-ai-integration[bot] in #41882
feat(proxy): add LiteLLM_DailyGlobalSpend key-free rollup for the usage dashboard by @devin-ai-integration[bot] in #41324
fix(proxy): keep requested model guardrails and key disable_fallbacks on rate-limit fallback by @devin-ai-integration[bot] in #41783
fix(bedrock): carry s3_endpoint_url and s3_region_name into file content downloads by @devin-ai-integration[bot] in #41138
ci(duplicate-check): let Codex reach GitHub from its sandbox by @ryan-crabbe-berri in #41883
fix(proxy): resolve model_group_alias to its target for /v1/models metadata by @devin-ai-integration[bot] in #41483
fix(proxy): reset sibling tpm/rpm counters when the shared rate limit window rolls over by @devin-ai-integration[bot] in #41838
feat(ui): link MCP Servers page to the user's connected MCP servers by @devin-ai-integration[bot] in #41888
fix(proxy): name the blocking guardrail in x-litellm-applied-guardrails by @devin-ai-integration[bot] in #41583
fix(responses): merge deployment litellm_params into native websocket response.create frames by @devin-ai-integration[bot] in #41881
feat: honor eager_input_streaming on Bedrock and Anthropic Claude tools by @devin-ai-integration[bot] in #41871
fix(otel): keep caller traceparent and tracestate on pass-through relays by @devin-ai-integration[bot] in #41786
ci: remove the dead Agent Shin triage workflows and scripts by @devin-ai-integration[bot] in #41894
fix: set vertex gemma-4-26b-a4b-it-maas context window to 262144 by @devin-ai-integration[bot] in #41887
refactor(rust): formalize legacy callback contract by @devin-ai-integration[bot] in #41885
fix(gemini): preserve candidates with finishReason and no content (#40477) by @devin-ai-integration[bot] in #41892
fix(team): apply team_member_budget updates to members still on the team default by @devin-ai-integration[bot] in #41347
fix(enterprise): resolve openai_moderations model at call time and default to omni-moderation-latest by @devin-ai-integration[bot] in #41895
fix(proxy): track project spend and enforce project budgets additively by @devin-ai-integration[bot] in #41354
fix(proxy): refuse config-owned keys on POST /config/update by @devin-ai-integration[bot] in #41868
test(integration): cover off-peak pricing on a live proxy by @devin-ai-integration[bot] in #41901
fix(proxy): requeue daily spend rows when the commit fails without the Redis buffer by @devin-ai-integration[bot] in #41878
fix(responses): emit typed streaming failure events by @zoroyihan7 in #40243
fix(bedrock): clamp maxTokens to the 16-token minimum for OpenAI GPT and xAI Grok models on Converse by @devin-ai-integration[bot] in #41870
chore(model_info): backfill reseller Gemini entries from provider catalogs and prune retired ids by @devin-ai-integration[bot] in #41902
feat(mcp): allowlist MCP client applications at the gateway by @devin-ai-integration[bot] in #41667
fix(schema): classify off_peak_pricing as a structured object in the model prices schema generator by @devin-ai-integration[bot] in #41847
fix(rate_limiter): render the 429 reset time in UTC as labelled by @devin-ai-integration[bot] in #41911
ci(issues): comment which release carries the fix when a pull request closes an issue by @ryan-crabbe-berri in #41910
feat(proxy): let team admins manage projects via team_admin_editable_team_fields by @devin-ai-integration[bot] in #41916
fix(bedrock): sign batch retrieve and cancel with deployment credentials when AWS_BEARER_TOKEN_BEDROCK is set by @devin-ai-integration[bot] in #41904
fix(websearch_interception): surface a failed search as a web_search_tool_result_error block and end the turn by @devin-ai-integration[bot] in #41905
fix(cost_calc): default fireworks cached input to the documented 50% discount when the map has no cache-read rate by @devin-ai-integration[bot] in #41917
fix(timing): anchor response duration and overhead at proxy receive time by @devin-ai-integration[bot] in #41891
feat(xai): add speech-to-text (Grok Voice Transcribe) via /v1/audio/transcriptions by @devin-ai-integration[bot] in #41914
feat(rust): add litellm-http client pool and inject it into the OCR route by @devin-ai-integration[bot] in #41897
feat(vertex_ai): stream Chirp speech-to-text over /v1/realtime by @devin-ai-integration[bot] in #41721
chore: consolidate CLAUDE.md into AGENTS.md by @devin-ai-integration[bot] in #41921
feat(proxy): serve the Claude Code gateway protocol under /claude_code_gateway by @devin-ai-integration[bot] in #34267
fix(responses): restore encrypted_content and apply affinity on the native WebSocket relay by @devin-ai-integration[bot] in #41893
feat(proxy): add RFC 8693 token exchange for IdP JWTs on the gateway token endpoint by @devin-ai-integration[bot] in #41485
fix(websearch): forward the deployment api_base to agentic follow-up calls on /v1/messages by @devin-ai-integration[bot] in #41918
fix(proxy): parse role_permissions where it is read by @yuneng-berri in #41924
fix(responses): announce message item before text events in the chat completions bridge by @devin-ai-integration[bot] in #41564
fix(proxy): unpin cost-map pricing copied into model_info and report pricing overrides by @devin-ai-integration[bot] in #41843
feat(websearch): let the model emit objective + multi-query search shapes by @adssoccer1 in #40399
fix(exceptions): keep internal_server_error as the public type of an upstream 500 by @devin-ai-integration[bot] in #41930
fix(proxy): dispatch llm_api_check moderation through during_call_hook by @devin-ai-integration[bot] in #41685
fix(policy_engine): deliver guardrail text rewrites on multi-choice, unfinished, and envelope-less streams by @devin-ai-integration[bot] in #41933
fix(anthropic): keep cache_control for Gemini targets on /v1/messages and normalize Anthropic ttl units by @devin-ai-integration[bot] in #41938
fix(guardrails): stop the Javelin api_version default leaking into Azure Content Safety by @devin-ai-integration[bot] in #41941
fix(alerting): clarify budget threshold messages by @gaurav-pandey-zocdoc in #39102
fix(cost): carry image and video input tokens through the Responses usage bridge (internal copy of #36887) by @devin-ai-integration[bot] in #41237
ci(unit): fail a hung test in 120s with a traceback instead of idling the shard to its step timeout by @devin-ai-integration[bot] in #41948
refactor(types): replace Any with proven types in 6 files by @devin-ai-integration[bot] in #41947
fix(proxy): register transcribe as a known provider for model grants by @yuneng-berri in #41926
feat(batches): support Mistral files/batches and per-page OCR batch cost tracking (internal copy of #40484) by @devin-ai-integration[bot] in #41934
fix(utils): reject an untranslatable tool_choice with a 400 instead of a 500 by @devin-ai-integration[bot] in #41234
fix(proxy): keep request metadata out of the cost tracking failure alert by @devin-ai-integration[bot] in #41950
fix(masker): memoize shared nodes and fail closed past the depth cap by @devin-ai-integration[bot] in #41952
fix(responses): drop tool_search and local_shell in the chat completions bridge by @devin-ai-integration[bot] in #41953
fix(rag): resolve registry stores on /v1/rag/ingest and reject providers without ingestion by @devin-ai-integration[bot] in #41940
fix(azure): send the resolved Entra ID token on image generation requests by @abhirup7 in #40147
fix(proxy): keep the raw client model out of spend logs for rejections outside the router by @devin-ai-integration[bot] in #41943
fix(cost): resolve dated openai/azure snapshots to their undated cost map entry by @devin-ai-integration[bot] in #41423
fix(proxy): /key/bulk_update writes only the fields each item carries by @devin-ai-integration[bot] in #41949
fix(cost): bill DeepSeek V4.1 Flash and V4 Pro at off-peak rates outside peak hours by @devin-ai-integration[bot] in #41960
test(cost): point dated snapshot tests at a date the cost map cannot carry by @devin-ai-integration[bot] in #41971
fix(proxy): refuse runtime writes to config-owned settings by @yuneng-berri in #41931
refactor(rust): centralize layered settings resolution by @devin-ai-integration[bot] in #41969
test(utils): isolate dated model fallback from pricing additions by @joshua-berri in #41975
test(response_metadata): anchor detailed-timing test on a fixed instant instead of wall clock by @devin-ai-integration[bot] in #41946
test(integration): literal request/response cost tracking suite on a test-owned cost map by @devin-ai-integration[bot] in #41328
chore(prices): sync Azure prices: 5 models, 5 deprecated by @berriai-litellm-provider-info-sync[bot] in #41966
feat(ocr): add Rust-only Textract and sign provider requests after host hooks by @devin-ai-integration[bot] in #41977
refactor(rust): use typed pyo3 APIs instead of getattr/import strings by @devin-ai-integration[bot] in #41981
fix(router): enforce model tpm limits against shared redis usage across replicas by @devin-ai-integration[bot] in #41915
feat(proxy): say when a stored setting is ignored because the config file owns it by @yuneng-berri in #41985
fix(otel v2): summarize embedding vectors as Langfuse observation output by @devin-ai-integration[bot] in #41982
fix(caching): scope automatic breakpoints to supported Claude transports by @tin-berri in #41920
fix(proxy): return 400 instead of 500 for /v1/responses without input by @devin-ai-integration[bot] in #41939
fix(llmguard): accept proxy async call types by @yucheng-berri in #41787
fix(model_prices): drop anthropic deprecation floors and correct azure gpt-4.1-nano retirement date by @devin-ai-integration[bot] in #41964
fix(ui): show user attribution in Top Virtual Keys usage tables by @devin-ai-integration[bot] in #40729
chore(prices): sync OpenRouter prices: 172 models, 2 new by @berriai-litellm-provider-info-sync[bot] in #41833
fix(ocr): set DeepSeek OCR sampling defaults by @devin-ai-integration[bot] in #41992
ci: remove auto-merge-price-sync workflow, the Devin sync automation merges price PRs by @devin-ai-integration[bot] in #41993
refactor(mcp): upgrade SDK2 while preserving legacy gateway behavior by @joshua-berri in #41718
revert(guardrails): drop the scoped request conversation and tools from post-call scans (#41220) by @devin-ai-integration[bot] in #41986
chore(prices): sync OpenRouter prices: 2 models by @berriai-litellm-provider-info-sync[bot] in #41996
test(unified_google_tests): use the Vertex global endpoint and retry 429s with backoff by @devin-ai-integration[bot] in #41995
fix(rust): refuse native routes in processes forked after the runtime started by @devin-ai-integration[bot] in #41987
fix(terraform): unlink the registry docs entries that 404 on click by @shivamrawat1 in #42003
fix(otel v2): map Responses API output onto the Langfuse generation output by @devin-ai-integration[bot] in #41991
test(e2e): stop the config suite locking itself out of the shared proxy by @yuneng-berri in #42008
chore: bump litellm-enterprise 0.1.68 -> 0.1.69, litellm-proxy-extras 0.4.99 -> 0.4.100 by @yuneng-berri in #42025
fix(proxy): close the config-ownership gaps QA found in the settings store by @yuneng-berri in #42009
fix(proxy): block project requests when max_budget is 0 by @ryan-crabbe-berri in #41997
fix(auth): inherit org alias, budget and rate limits for JWT and team-linked keys by @devin-ai-integration[bot] in #41681
feat(batches): run hosted_vllm batches inside LiteLLM by @devin-ai-integration[bot] in #41942
fix(proxy): forward stream response attributes through the hook boundary and merge logged applied_guardrails by @devin-ai-integration[bot] in #42027
feat(ui): configure web search interception from the Admin UI by @yuneng-berri in #42007
fix(azure): drop tool_choice when the request has no tools (internal copy of #41781) by @devin-ai-integration[bot] in #42031
fix(azure): tool choice without tools causes 400 by @dsteeley in #41781
test(mcp): verify scoped execution and OAuth credential isolation by @joshua-berri in #41731
test(e2e): cover MCP OAuth SSO and cold restart persistence by @devin-ai-integration[bot] in #41909
fix(proxy): estimate auto-router baseline costs from durable cache history by @tin-berri in #41177
fix(auto-router): show heuristic v2 score estimates in routing details by @tin-berri in #42001
feat(router): add maintained Fuse model and harness presets by @tin-berri in #41617
refactor(rust): rename legacy callback adapter crate by @devin-ai-integration[bot] in #42037
Revert "test(mcp): verify scoped execution and OAuth credential isolation" by @mateo-berri in #42039
revert(e2e): remove MCP OAuth SSO and cold restart tests by @mateo-berri in #42040
test(mcp): cover SDK redirect compatibility by @joshua-berri in #42033
feat(ui): report whether the serving proxy has applied web search interception by @yuneng-berri in #42042
fix(test): unbreak the integration-cost and proxy_e2e_anthropic_messages CircleCI jobs on main by @devin-ai-integration[bot] in #42048
chore(prices): sync OpenRouter prices: 2 models by @berriai-litellm-provider-info-sync[bot] in #42006
feat(otel v2): opt-in llm_only span scope for Langfuse destinations and the operator Langfuse exporter by @devin-ai-integration[bot] in #41740
test(e2e): point the Nova Sonic realtime test at nova-2-sonic by @yuneng-berri in #42053
fix(team): emit audit events for member_delete and role changes and carry the final roster on team create by @devin-ai-integration[bot] in #41840
chore(prices): sync OpenRouter prices: 5 models by @berriai-litellm-provider-info-sync[bot] in #42058
chore(prices): sync OpenRouter prices: 2 models by @berriai-litellm-provider-info-sync[bot] in #42063
chore: rebuild Admin UI bundle from main (build kXnLzJ6ylsRPmgSkCkCKM) by @yuneng-berri in #42054
test(mcp): migrate the mcp test helpers to the mcp 2.x MCPServer API by @yuneng-berri in #42075
test(logging): add autorouter estimate keys to the GCS pub/sub spend-log golden by @yuneng-berri in #42076
test(batches): skip the Bedrock GovCloud batch e2e until its secrets are provisioned by @yuneng-berri in #42081

New Contributors

@clonylu made their first contribution in #41203
@zoroyihan7 made their first contribution in #40243
@adssoccer1 made their first contribution in #40399
@gaurav-pandey-zocdoc made their first contribution in #39102
@abhirup7 made their first contribution in #40147

Full Changelog: v1.103.0-dev.2...v1.103.0-rc.1