BerriAI/litellm v1.103.0-dev.1 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.103.0-dev.1

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.103.0-dev.1/cosign.pub \
  ghcr.io/berriai/litellm:v1.103.0-dev.1

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

fix(responses): translate the reasoning object into a chat-completion reasoning effort by @joshgarnett in #36363
fix(proxy): bound tool and guardrail index create_many by the spend-log statement budgets by @devin-ai-integration[bot] in #40561
fix(mcp): require admission for delegated OAuth by @joshua-berri in #40923
fix(logging): log one bounded summary for a burst of timed-out LoggingWorker callbacks by @devin-ai-integration[bot] in #40912
fix(fireworks): resolve short model names to long cost map keys by @devin-ai-integration[bot] in #40929
ci: remove main branch source guard by @yuneng-berri in #40172
chore(ci): promote internal staging to main by @yuneng-berri in #40942
fix(spend_logs): store litellm_call_id and match it in request_id lookups by @mateo-berri in #39068
fix(auth): refresh lite login session token grants from the live user and team rows by @ryan-crabbe-berri in #40657
feat(bedrock): support file delete and list for S3-backed managed files by @mateo-berri in #39836
fix(proxy): gate the webhook test alert on proxy admins by @ryan-crabbe-berri in #40814
fix(ui): hide admin write-form tabs on the models page from view-only admins by @mateo-berri in #38867
fix(anthropic-adapter): surface mid-stream provider errors as Anthropic error events by @devin-ai-integration[bot] in #33352
docs(github): add an Affected release section to the PR template by @mateo-berri in #40618
docs(e2e): ban unit tests under tests/e2e by @yassin-berriai in #33852
fix(router): preserve Azure Entra ID params in reusable credentials by @devin-ai-integration[bot] in #40889
docs(user endpoints): remove unsupported soft_budget param from user docstrings by @shivamrawat1 in #36585
feat(friendli): auto-sync Friendli model metadata into price registry by @Lee-Si-Yoon in #35918
build(deps): bump smol-toml to 1.8.0 to clear GHSA-7w5x-hrqm-74c2 in osv-scan by @devin-ai-integration[bot] in #40478
fix(bedrock_mantle): price GovCloud regions from the regional cost row and accept region-prefixed model names by @mateo-berri in #39846
chore(ci): remerge internal staging by @yuneng-berri in #40943
test(auth): freeze the cache clock in auth prefetch tests by @devin-ai-integration[bot] in #40996
feat(jwt): allow virtual_key_claim_field per issuer by @devin-ai-integration[bot] in #40927
fix(cost): bill cached realtime audio tokens at the audio cache-read rate by @devin-ai-integration[bot] in #40627
perf(logging): skip correlation contextvar stamping when request_correlation_in_logs is off by @devin-ai-integration[bot] in #41054
feat(pricing): add azure gpt-chat-latest rates and drop retired friendliai llama-3.1 entries by @devin-ai-integration[bot] in #40976
build(deps): re-suppress GHSA-h7x2-h6g9-p789 in osv-scan on main, mlflow still has no fixed release by @devin-ai-integration[bot] in #41104
fix(otel): cap per-index OpenInference message attributes span-wide by @devin-ai-integration[bot] in #40562
feat(proxy): add general_settings.allowed_file_extensions for /v1/files uploads by @devin-ai-integration[bot] in #41106
fix(proxy): forward provider request id headers on mapped error responses by @devin-ai-integration[bot] in #40925
fix(router): name the all-deployments-in-cooldown error on 429 responses by @devin-ai-integration[bot] in #40995
fix(ui): show the team alias on the model info page and in its raw JSON by @devin-ai-integration[bot] in #40992
feat(proxy): honor LITELLM_DISABLE_ACCESS_LOG_PATHS to drop noisy uvicorn access log lines by @devin-ai-integration[bot] in #41096
fix(prometheus): label pre-call rate limit failures with the resolved api_provider by @devin-ai-integration[bot] in #41059
perf(proxy): serialize /model/info listing once with orjson by @devin-ai-integration[bot] in #41114
fix(utils): stop wrapper_async submitting the sync success handler twice by @devin-ai-integration[bot] in #41115
fix(redis): log a timeout streak once per interval instead of one line per cache call by @devin-ai-integration[bot] in #40817
fix(router): record flat retry attempts and cap retries from attempted_retries by @devin-ai-integration[bot] in #40930
refactor(prometheus): source PROXY_LLM_PROVIDER_FALLBACK from litellm.constants by @devin-ai-integration[bot] in #41118
fix(proxy): hide default credentials login hint when UI_PASSWORD is set by @devin-ai-integration[bot] in #41107
fix(cli): show routed models and session stats for LLM API keys by @tin-berri in #41116
fix(bedrock/realtime): propagate deferred Nova Sonic stream failures to the router by @devin-ai-integration[bot] in #41064
fix(proxy): keep org admins' own team memberships in other orgs visible on team list by @devin-ai-integration[bot] in #41086
feat(model_info): provider-scoped fill_missing_for_providers backfill from fallback generalization rules by @devin-ai-integration[bot] in #41093
fix(auth): load team membership once per request and skip prisma on an L1 hit by @devin-ai-integration[bot] in #41102
refactor(harness): expand independent trace coverage by @yujonglee-berri in #41120
fix(proxy): release max_parallel_requests slot when a realtime session ends without LLM callbacks by @devin-ai-integration[bot] in #41113
fix(router): cool down team deployments on 429 when a sibling serves the same public model by @devin-ai-integration[bot] in #40991
fix(ui): move tags typed into key metadata JSON into the Tags field by @devin-ai-integration[bot] in #41023
fix(ui): let team admins grant a team all proxy models by @ryan-crabbe-berri in #40196
chore(lint): graduate 12 rules from the strict-gate ratchet by @HUAHAODIA in #41048
test: add dedicated CircleCI integration contract foundation by @yuneng-berri in #41066
fix(utils): keep litellm params out of provider request bodies by @devin-ai-integration[bot] in #41018
fix(openai): keep extra_headers out of the chat request body on the httpx handler path by @devin-ai-integration[bot] in #41141
test: cover persisted updates and warmed authorization policies by @yuneng-berri in #41070
fix(proxy): resolve x-litellm-call-id from response metadata when routes omit call_id by @devin-ai-integration[bot] in #41056
chore(prices): sync Vertex AI prices: 14 models by @berriai-litellm-provider-info-sync[bot] in #40955
ci(codeql): exclude noisy Python quality queries by @yuneng-berri in #41142
feat(proxy): predict prompt-cache costs across deployments by @tin-berri in #40877
fix(prompt_security): keep polling file sanitization through non-terminal statuses by @devin-ai-integration[bot] in #41131
fix(health): skip background health check DB writes when the latest-row read fails by @devin-ai-integration[bot] in #41145
feat(model_armor): logging_only mode scans completed streams after delivery by @devin-ai-integration[bot] in #40702
fix(bedrock guardrails): derive contextual grounding source and query from plain messages by @devin-ai-integration[bot] in #41132
fix(cli): drop enum.StrEnum so the CLI imports on Python 3.10 by @devin-ai-integration[bot] in #41046
fix(responses): route mid-stream error events through exception_type so content_policy_fallbacks fire by @devin-ai-integration[bot] in #40988
fix(cost): bill gemini-embedding-2 per token and stop double charging audio by @devin-ai-integration[bot] in #41157
test: bind management E2E callers and isolate JWT actors by @yuneng-berri in #40892
fix(headroom): protect cache_control-marked rows anywhere in history by @rad-p44 in #40315
test: add strict stateless provider replay identity by @yuneng-berri in #41149
fix(ci): test checked-out model pricing in unit jobs by @yuneng-berri in #41181
fix(guardrails): write per-message guardrail rewrites back onto Responses input items by @devin-ai-integration[bot] in #40939
fix(proxy): log the provider usage on deferred /v1/messages calls and price cache writes without a creation rate by @devin-ai-integration[bot] in #41172
fix(guardrails): record not_run evaluation when scoping leaves nothing to scan by @yucheng-berri in #39050
fix(responses): preserve provider affinity by @AaronHowell in #40228
fix(sdk): keep body and proxy headers on BadRequestError mapped from a litellm_proxy 400 by @devin-ai-integration[bot] in #40994
fix(responses): hoist Codex additional_tools input items into the chat bridge tools by @devin-ai-integration[bot] in #40989
fix(router): honor team and key provider weights by @tin-berri in #41072
test(e2e): verify streamed answers and tool continuation by @yuneng-berri in #41194
fix(cli): label router costs and simplify the routed-model header by @tin-berri in #41186
test(spend): reconcile concurrent requests and daily activity by @yuneng-berri in #41188
fix(guardrails): scan the Anthropic top-level system prompt and tool_use arguments by @devin-ai-integration[bot] in #40984
fix(router): count num_retries_per_request across fallback hops by @devin-ai-integration[bot] in #41191
fix(bedrock): grant rerank, retrieve, agent, and agentcore actions in the web identity session policy by @devin-ai-integration[bot] in #41168
fix(vertex-live): bill Gemini Live sessions end to end (internal copy of #37075) by @devin-ai-integration[bot] in #40915
fix(health): resolve litellm_credential_name in realtime health checks by @devin-ai-integration[bot] in #41173
feat(proxy): unified custom_key_policy hook for key generate, update and regenerate by @devin-ai-integration[bot] in #40921
fix(proxy): enforce custom_key_update policy on /key/regenerate by @devin-ai-integration[bot] in #40695
fix(router): preserve session model choice within each complexity tier by @tin-berri in #41174
test(pricing): let synced GovCloud Bedrock rows cite the AWS price list by @devin-ai-integration[bot] in #41263
docs(github): ask for interactive coding-tool proof in the PR template by @devin-ai-integration[bot] in #41257
feat(proxy): add POST /management/v1/users/bulk for batched user and team membership creation by @devin-ai-integration[bot] in #41028
fix(credentials): answer 409 on a credential name collision, make Terraform adoption opt-in by @ryan-crabbe-berri in #40917
feat(proxy): add POST /management/v1/users/bulk_delete and POST /management/v1/teams/{team_id}/members/bulk_delete by @devin-ai-integration[bot] in #41039
fix(proxy): list directly assigned team models in model access errors by @devin-ai-integration[bot] in #41256
feat(auto-router): allow opted-in team members to manage their routers by @tin-berri in #41175
build(rust-bridge): add typed _native stub and validate it with mypy.stubtest by @devin-ai-integration[bot] in #41180
feat(guardrails): add new upstream presidio pii entities including german set by @MvdB in #36775
fix(responses): filter bridged kwargs like the native Responses path by @devin-ai-integration[bot] in #41144
test(e2e): cover the reliability retry, cooldown, fallback, and routing-strategy cells by @mateo-berri in #39857
fix(anthropic): add the per-turn-control beta when a message carries output_config by @devin-ai-integration[bot] in #41189
fix(router): bind per-request routing_strategy override selectors to the request's callbacks by @devin-ai-integration[bot] in #41178
feat(proxy): bind JWT claims to registered agents via agent_id_jwt_field by @devin-ai-integration[bot] in #40904
fix(proxy): enforce organization budgets when max_budget is 0 by @yassin-berriai in #41271
fix(alerting): send llm_exceptions Slack alert for 5xx HTTPException and ProxyException by @devin-ai-integration[bot] in #41125
fix(headroom): protect the cached prefix through the last cache_control breakpoint by @devin-ai-integration[bot] in #41161
fix(utils): cache custom HuggingFace tokenizers across /utils/token_counter requests by @devin-ai-integration[bot] in #41216
fix(router): keep weighted routing when a deployment id equals a model_name by @devin-ai-integration[bot] in #41156
feat(router): add capability classifier as Fuse foundation by @tin-berri in #41270
fix(proxy): keep access-group raw SQL writes on the writer while writer_unavailable is stale by @devin-ai-integration[bot] in #41283
fix(prometheus): count 401 auth failures in litellm_proxy_failed_requests_metric by @devin-ai-integration[bot] in #41170
test: drop tests that pin vendor facts and add the CLAUDE.md rule by @devin-ai-integration[bot] in #41269
fix(proxy): run the remaining inline token counts off the event loop by @devin-ai-integration[bot] in #40262
fix(proxy): log blocked streaming guardrail responses as failures, not success by @devin-ai-integration[bot] in #40191
feat(proxy): add tpd_limit (tokens per day) for batch submissions by @devin-ai-integration[bot] in #40997
fix(proxy): reconcile budget reservation before enqueuing spend to the DB by @devin-ai-integration[bot] in #40310
fix(xai): stop sending web_search_options to xAI's retired Live Search path by @devin-ai-integration[bot] in #38278
feat(terraform): add tpm_limit, rpm_limit, budget_duration, allowed_models to litellm_team_member_add by @devin-ai-integration[bot] in #38682
fix(rerank): bill Vertex search_units from input records and give every rerank response a unique id by @devin-ai-integration[bot] in #35180
fix(router): stop counting caller-set timeout 408s toward deployment cooldown by @devin-ai-integration[bot] in #41230
feat(router): add Fuse V2 classifier after capability forecasting by @tin-berri in #41272
fix(proxy): keep client User-Agent on auth failure spend logs by @devin-ai-integration[bot] in #41291
fix(proxy): reset budgets by decrementing pre-reset spend instead of zeroing rows by @devin-ai-integration[bot] in #41279
fix(xai): honor nested web_search filters on the xAI Responses API by @devin-ai-integration[bot] in #38268
fix(router): stop registering a caller-supplied credential as a router deployment by @yassin-berriai in #41289
fix(router): accept custom_provider_map providers before the first completion call by @devin-ai-integration[bot] in #41300
fix(proxy): return 400 instead of 500 for lone surrogate escapes in request body by @devin-ai-integration[bot] in #41297
fix(langsmith): keep events appended during an in-flight flush instead of clearing them by @devin-ai-integration[bot] in #41288
fix(logging): track spend for streams a deployment hook converted to non-streaming by @devin-ai-integration[bot] in #41171
fix(bedrock): sanitize client tool_call ids to Bedrock toolUseId constraints by @devin-ai-integration[bot] in #40872
fix(passthrough): attribute Vertex passthrough successes to the resolved router deployment by @devin-ai-integration[bot] in #41307
feat(ui): persist Models table search, filters, sort and page in the URL by @devin-ai-integration[bot] in #41296
fix(jwt-auth): scope JWT key mappings by issuer to prevent cross-issuer collisions by @yassin-berriai in #41281
feat(openai): add openai_system_messages_first to put system messages first for prompt caching by @devin-ai-integration[bot] in #41304
feat(ui): add custom request headers to the API Playground by @devin-ai-integration[bot] in #41309
feat(cli): sync Codex /model picker from proxy /v1/models in lite codex by @devin-ai-integration[bot] in #40476
chore: bump litellm-enterprise 0.1.67 -> 0.1.68, litellm-proxy-extras 0.4.97 -> 0.4.98, litellm 1.102.0 -> 1.103.0 by @yuneng-berri in #41321
feat: add aihubmix provider pricing entries by @IToSSc in #41179
feat(auto-router): add per-model Fast mode toggle by @tin-berri in #41282
fix(proxy): include litellm_call_id in LLM API exception logs by @devin-ai-integration[bot] in #41205
fix(proxy): keep yaml pass-through endpoints visible to auth after db overlay by @devin-ai-integration[bot] in #41303
fix(proxy): resolve router_settings.model_group_alias before key/team model auth by @devin-ai-integration[bot] in #41308
fix(ui): block usage export and flag the range when a spend page fails by @ryan-crabbe-berri in #41294
fix(vertex_ai): bill Gemini Omni Interactions usage and Veo sampleCount on passthrough by @devin-ai-integration[bot] in #41322
fix(proxy): honor LITELLM_LOG for uvicorn and proxy extras loggers by @devin-ai-integration[bot] in #41306
fix(cost): price native Responses WebSocket turns at their returned service_tier by @devin-ai-integration[bot] in #41318
fix(proxy): key model rpm/tpm override takes precedence over team model limit by @devin-ai-integration[bot] in #41302
fix(proxy): track per-member organization spend by @devin-ai-integration[bot] in #41255
feat(proxy): add /nvidia_nim passthrough route for NIM object detection and OCR /v1/infer by @devin-ai-integration[bot] in #41316
feat(model_info): add provider-neutral Gemini 2.5+ chat baseline fallback generalization by @devin-ai-integration[bot] in #41320
fix(spend): sum multi-round session duration in logs UI by @devin-ai-integration[bot] in #35388
feat(router): limit unlicensed Capability and Fuse v2 routers to one each by @tin-berri in #41326
fix(guardrails): resolve caller identity from metadata buckets in custom code guardrail by @devin-ai-integration[bot] in #41126
fix(e2e): onboard dashboard users through invitations by @yuneng-berri in #41319
feat(guardrails): add Microsoft Agent 365 MCP tool-call guardrail by @yucheng-berri in #38241
test: drop remaining tests that pin cost-map vendor facts by @devin-ai-integration[bot] in #41298
feat(ui): show average response time per model in usage model activity by @devin-ai-integration[bot] in #41313
fix(proxy): preserve Anthropic pricing modifiers in router savings by @tin-berri in #41341
feat(guardrails): support pre_call and during_call modes for llm_as_a_judge by @devin-ai-integration[bot] in #41128
fix(gemini): propagate the provider's modelVersion to the response model by @devin-ai-integration[bot] in #41338

New Contributors

@joshgarnett made their first contribution in #36363
@HUAHAODIA made their first contribution in #41048
@berriai-litellm-provider-info-sync[bot] made their first contribution in #40955
@rad-p44 made their first contribution in #40315
@AaronHowell made their first contribution in #40228
@MvdB made their first contribution in #36775
@IToSSc made their first contribution in #41179

Full Changelog: v1.102.0-rc.1...v1.103.0-dev.1