BerriAI/litellm v1.103.0-dev.2 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.103.0-dev.2

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.103.0-dev.2/cosign.pub \
  ghcr.io/berriai/litellm:v1.103.0-dev.2

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

fix(fireworks-ai): bill cache-write, reasoning and audio tokens via the shared cost calculator by @devin-ai-integration[bot] in #41339
feat(guardrails): singulr v2 API contract with logging_only, pre_mcp_call and post_mcp_call by @yucheng-berri in #41329
ci(image-scan): ignore zlib CVE-2026-85091 until Wolfi ships the fix by @yuneng-berri in #41353
feat(e2e): reuse exact provider responses for 24 hours by @yuneng-berri in #41346
fix(xai): keep 'instructions' on the xAI Responses API so system messages survive web search by @devin-ai-integration[bot] in #38254
feat(ui): configure capability and Fuse v2 classifiers by @tin-berri in #41315
fix(anthropic): tolerate message_delta events without usage when streaming by @devin-ai-integration[bot] in #41336
test(router): ignore deployment-selection logs in the fallback log assertion by @yuneng-berri in #41358
test(proxy): assert budget resets decrement the cleared spend by @yuneng-berri in #41359
fix(e2e): expect models filters to persist after reload by @yuneng-berri in #41348
fix(e2e): record cookie-setting provider responses and keep prompt-caching tests live by @yuneng-berri in #41366
fix(responses): recount tokens when a streamed response completes without usage by @devin-ai-integration[bot] in #41337
fix(ui): simplify Capability and Fuse advanced routing options by @tin-berri in #41371
fix(mcp): authorize JWT OAuth credential persistence by @joshua-berri in #41314
feat(router): stream shadow traffic and fan out silent_model to multiple targets by @devin-ai-integration[bot] in #41368
perf(content_filter): scan a bounded window per streamed chunk by @devin-ai-integration[bot] in #41407
fix(proxy): hide model allowlist from client-facing model access denied errors by @devin-ai-integration[bot] in #41310
feat(http): opt-in outbound HTTP/2 for httpx clients by @devin-ai-integration[bot] in #41268
refactor(rust): remove gateway, config, router, realtime, and Rust trace-parity instrumentation by @devin-ai-integration[bot] in #41432
fix(guardrails): don't add post_call output scan for MCP-only Presidio modes by @devin-ai-integration[bot] in #40571
chore(prices): sync Azure, Azure AI, Gemini, OpenAI, Bedrock, Together AI, Fireworks and Vertex prices: 278 models, 59 new, 30 deprecated by @berriai-litellm-provider-info-sync[bot] in #41154
fix(rag): forward retrieval_filter from retrieval_config to vector store search by @devin-ai-integration[bot] in #34427
refactor(rust): extract auth and cache crates by @devin-ai-integration[bot] in #41464
fix(proxy): default litellm_trace_id to the OTel server span trace id by @devin-ai-integration[bot] in #41386
chore(codeowners): add ryan and kerry as owners of the cost map by @devin-ai-integration[bot] in #41333
fix(responses): guard empty-choices chunks in the Responses API streaming bridge by @devin-ai-integration[bot] in #34455
chore(prices): sync Google Gemini prices: 22 models by @berriai-litellm-provider-info-sync[bot] in #41457
fix(bedrock): forward userContext in Knowledge Base Retrieve requests by @devin-ai-integration[bot] in #41475
ci(rust): split rust jobs, use nextest and Swatinem/rust-cache by @devin-ai-integration[bot] in #41480
fix(fireworks_ai): flatten dict-form reasoning_effort to its effort string by @devin-ai-integration[bot] in #41335
fix(proxy): never forward the LiteLLM virtual key to Anthropic on the /anthropic passthrough by @devin-ai-integration[bot] in #41340
fix(proxy): rename AWS Secrets Manager secret when key alias changes by @devin-ai-integration[bot] in #41468
feat(otel): promote nested request metadata keys to litellm.metadata.* span attributes by @devin-ai-integration[bot] in #41462
fix(proxy): sync AWS Secrets Manager on body-less key regenerate and key alias changes by @devin-ai-integration[bot] in #41458
fix(http_handler): keep a handler alive while a response it issued is still reading by @max-sixty in #34829
fix(bedrock): make prompt caching work on the Nova InvokeModel route by @devin-ai-integration[bot] in #41343
ci(migrations): flag defaulted ADD COLUMN on request-log tables by @devin-ai-integration[bot] in #41460
feat(prometheus): add customer (end_user) budget gauges by @devin-ai-integration[bot] in #41472
fix(otel): drop None metric and event attributes before OTLP export by @devin-ai-integration[bot] in #36815
fix(anthropic): carry the served model from message_start onto stream chunks by @devin-ai-integration[bot] in #41446
fix(models): rolling registry audit: Gemini latest aliases, Nova cache pricing, OpenRouter/Together sync, Mistral GLM 5.3, Azure snapshots, Grok caching by @devin-ai-integration[bot] in #41112
fix(router): count TPM/RPM usage before building rate-limit headers by @devin-ai-integration[bot] in #41474
feat(guardrails): release buffered stream chunks after each passing scan by @devin-ai-integration[bot] in #41425
fix!: re-check budget on router fallback targets by @runjivu in #41379
refactor(ocr): move file preparation from the python bridge into litellm-core by @devin-ai-integration[bot] in #41489
feat(s3): add s3_log_prompts_only option to log prompts without responses by @devin-ai-integration[bot] in #41327
feat(team): team-level model_max_budget with key-level overrides by @devin-ai-integration[bot] in #41330
feat(keys): filter /key/list by active, expired, revoked or deleted status and serve deleted keys from /key/info by @devin-ai-integration[bot] in #41311
feat(proxy): expose lifetime total_spend on virtual keys by @devin-ai-integration[bot] in #41403
fix(proxy): release completed max-parallel slots promptly by @elifozdamar in #40843
feat(ui): accept ssh clone urls when registering a skill by @devin-ai-integration[bot] in #35418
fix(prices): dedupe Nova cache_read_input_token_cost keys left by a text merge by @yuneng-berri in #41496
fix(otel): propagate W3C trace context on HTTP and WebSocket passthrough by @devin-ai-integration[bot] in #40669
fix(proxy): remove duplicate user budget hook that 429'd zero-cost models by @devin-ai-integration[bot] in #41345
test(logging): pick this test's own records out of the shared log batch by @yuneng-berri in #41487
test(together_ai): move request-shape checks to the mapped file, drop the live ones by @yuneng-berri in #41360
feat(ui): shared URL-state layer for tables and tabs by @ryan-crabbe-berri in #41331
feat(e2e): make the provider cache reusable across builds and mount Bedrock behind it by @yuneng-berri in #41402
fix(mcp): fail closed on missing upstream credentials by @joshua-berri in #41364
feat(rust): scaffold Redis cache crate by @devin-ai-integration[bot] in #41501
fix(dashscope): forward reasoning_effort to the provider by @devin-ai-integration[bot] in #37506
fix(proxy): carry litellm_call_id through endpoint specific error logs and failure responses by @devin-ai-integration[bot] in #41356
fix(proxy): retry rate-limit fallbacks from a pristine request snapshot by @devin-ai-integration[bot] in #40596
fix(gemini): map minimal thinking to low for Gemini 3.7 and 3.8 Flash by @devin-ai-integration[bot] in #41201
fix(proxy): stop forwarding LiteLLM credential headers on Bedrock agent-runtime passthrough by @devin-ai-integration[bot] in #41504
fix(streaming): estimate interrupted Anthropic stream usage from reasoning_content by @devin-ai-integration[bot] in #41503
fix(azure_ai): route Responses API to native /openai/v1/responses for Foundry Models by @devin-ai-integration[bot] in #33856
fix(proxy): show all model groups to proxy admins in /model_group/info by @devin-ai-integration[bot] in #41094
feat(proxy): let proxy admins choose which team fields team admins may edit by @ryan-crabbe-berri in #39996
fix(bedrock): neutralize orphaned tool blocks instead of raising or injecting a dummy tool (internal copy of #31400) by @devin-ai-integration[bot] in #41513
feat(ui): persist organizations and projects list, detail tab and key table state in the URL by @ryan-crabbe-berri in #41445
fix(bedrock_mantle): accept and forward verbosity on gpt-5.x chat completions by @devin-ai-integration[bot] in #41509
test: cover database transactions and persisted accounting by @yuneng-berri in #41073
test: provider wire contracts, streaming and recovery by @yuneng-berri in #41075
fix(mcp): count admin static headers as api_key credential slots by @devin-ai-integration[bot] in #41514
ci: auto-merge provider-info-sync PRs when CI, Greptile and Bugbot are clean by @devin-ai-integration[bot] in #41494
feat(rust): add standalone framing crate by @devin-ai-integration[bot] in #41500
fix(proxy): enforce tag budgets for tags added by guardrails by @devin-ai-integration[bot] in #40842
fix(utils): run post-call deployment hook on converted chat streams by @devin-ai-integration[bot] in #41495
fix(e2e): bind provider-cache recordings to the deployment's test, not the serving process by @yuneng-berri in #41520
test: add extension and browser integration contracts by @yuneng-berri in #41078
fix(logging): scan each log record once and collapse base64 payloads before the secret regex by @devin-ai-integration[bot] in #40934
fix(spend_tracking): attribute router-rejected requests to the model group provider by @devin-ai-integration[bot] in #41507
feat(router): discover token limits for hosted OpenAI-compatible models by @devin-ai-integration[bot] in #41508
feat(proxy): let team admins edit rpm_limit and max_budget when enabled by @ryan-crabbe-berri in #41525
fix(otel): fit per-index OpenInference messages to the span's remaining attribute budget by @devin-ai-integration[bot] in #41498
test(aws): verify rotated secret value by @yuneng-berri in #41524
test(e2e): read a deleted key back as deleted, not as a 404 by @yuneng-berri in #41551
fix(otel v2): map the caller's Langfuse user, session and tags onto the root and generation spans by @devin-ai-integration[bot] in #41140
test: fix seven tests left stale by #41311, #41337, #39996, #41310, #41289 and #41315 by @yuneng-berri in #41527
test(budgets): cover management null handling by @yuneng-berri in #41563
test(e2e): drop the auto-router select "opens below" spec by @yuneng-berri in #41568
fix(guardrails): stream Prompt Security post_call redactions in incremental_diff mode by @devin-ai-integration[bot] in #41558
fix(guardrails): give post-call scans the scoped request conversation and tools by @devin-ai-integration[bot] in #41220
feat(openrouter): add stealth/union-alpha to the model cost map by @devin-ai-integration[bot] in #41576
test(management): cover project authorization lifecycle by @yuneng-berri in #41573
feat(rust): map Anthropic Messages transformations by @yujonglee-berri in #41531
fix(e2e): clear the three standing errors in the scheduled Buildkite suite by @yuneng-berri in #41616
refactor(rust_bridge): declarative route catalog and shared runtime selection by @devin-ai-integration[bot] in #41479
fix(mock_completion): keep the resolved provider so router custom pricing resolves for azure_ai deployments by @devin-ai-integration[bot] in #41623
fix(mcp): restrict health discovery to virtual key grants by @joshua-berri in #41609
fix(mcp): preserve request-selected guardrails during tool execution by @joshua-berri in #41619
refactor(ocr): mirror Python provider layout and preserve tests by @yujonglee-berri in #41550
test(fireworks_ai): stop pinning vision support on minimax-m3 by @devin-ai-integration[bot] in #41627
perf(spend_tracking): index LiteLLM_SpendLogs by (api_key, startTime) by @etiennechabert in #37983
fix(proxy): reject non-string model with 400 and log its spend as unknown-model by @devin-ai-integration[bot] in #41633
test(together_ai): stop pinning successor deprecation status by @devin-ai-integration[bot] in #41635
chore(prices): sync Together AI prices: 6 models, 6 deprecated [sync failed: Google Gemini] by @berriai-litellm-provider-info-sync[bot] in #41570
fix(budgets): page end-user cache invalidation after a budget reset by @ryan-crabbe-berri in #41488
chore: bump litellm-proxy-extras 0.4.98 -> 0.4.99 by @yuneng-berri in #41659
fix(tests): resolve the integration support package without run.py's PYTHONPATH by @yuneng-berri in #41373
fix(ui): keep untimed guardrail entries on the request lifecycle by @yuneng-berri in #41374
fix(anthropic-bridge): convert mid-conversation system turns to user turns on /v1/messages to chat completions by @devin-ai-integration[bot] in #41493
fix(bedrock): support aws-sdk-bedrock-runtime 0.10/0.11 in Bedrock Realtime by @devin-ai-integration[bot] in #41542
feat(cli): deprecate the litellm-proxy entrypoint in favour of lite by @devin-ai-integration[bot] in #41673
fix(scim): align pagination count validation with RFC 7644 by @zachbernstein-sdx in #41444
fix(bedrock): never emit Converse cachePoint for OpenAI-family models by @devin-ai-integration[bot] in #41419
fix(images): stop forwarding the raw image[] and mask[] form keys by @mateo-berri in #39512
feat(management_v1): bulk update team member budgets by @ryan-crabbe-berri in #41632
refactor(rust): extract provider translations by @devin-ai-integration[bot] in #41690
feat(cli): rename lite autoroute up/down to start/stop, keeping the old names as deprecated aliases by @devin-ai-integration[bot] in #41672
fix(responses): keep the addressed response id off bridged provider requests by @devin-ai-integration[bot] in #41689
fix(license): let a wildcard allowed_features license grant the auto_router feature by @devin-ai-integration[bot] in #41684
fix(ui): list every provider in the cache leakage by-model table by @devin-ai-integration[bot] in #40875
fix(team): keep a forked member budget's reset window and audit bulk member budget writes by @ryan-crabbe-berri in #41686
feat(proxy): add TypeSafe AI Jev evaluate passthrough with registry-priced spend tracking by @devin-ai-integration[bot] in #41607
test(e2e): cover bedrock batch file upload and create in the us-gov-west-1 partition by @devin-ai-integration[bot] in #41536
feat(grafana): add all-metrics dashboard and fix stale dashboard_v2 gauges by @devin-ai-integration[bot] in #41578
fix(cost): price Azure PTU spillover requests at standard token rates by @devin-ai-integration[bot] in #41569
build(deps): bump soupsieve to 2.9.2 to clear the osv-scan advisories by @devin-ai-integration[bot] in #41703
fix(fireworks_ai): restore supports_vision on minimax-m3 in the cost map by @devin-ai-integration[bot] in #41699
feat(policy_engine): explicit priority for policy attachment execution order by @devin-ai-integration[bot] in #41571
feat(router): add TypeSafe Jev as a complexity router classifier by @devin-ai-integration[bot] in #41615
ci(auto-merge): stop requiring Greptile and Bugbot on price sync pull requests by @devin-ai-integration[bot] in #41678
fix(ui): persist disabling cache control injection points on model update by @devin-ai-integration[bot] in #40632
fix(bedrock): gate Invoke tool search on the model map for Opus 4.8 and gen 5 Claude by @devin-ai-integration[bot] in #41702
fix(proxy): propagate db model renames to key, team, org, project and user model allowlists by @devin-ai-integration[bot] in #41694
test: delete unit-test assertions that pin cost-map prices, limits and deprecation dates by @devin-ai-integration[bot] in #41443
chore(tests): remove commented-out hf, petals and vertex ai completion blocks by @devin-ai-integration[bot] in #41640
chore(openai): drop commented-out legacy cost_per_token implementation by @devin-ai-integration[bot] in #41641
chore(proxy): delete deprecated unused litellm/proxy/_logging.py by @devin-ai-integration[bot] in #41642
chore(proxy): remove unreferenced performance_utils profiling module by @devin-ai-integration[bot] in #41643
test(streaming): remove commented-out retired-provider streaming tests by @devin-ai-integration[bot] in #41648
refactor(langfuse): remove unreachable langfuse v1 logging path by @devin-ai-integration[bot] in #41652
test: remove 15 fully commented-out test files that collect no tests by @devin-ai-integration[bot] in #41654
chore(tests): remove fully commented-out proxy test files and their CI entries by @devin-ai-integration[bot] in #41660
refactor(prometheus): remove unreferenced metric validators and pretty printers by @devin-ai-integration[bot] in #41661
chore(streaming): remove retired ai21/maritalk/baseten/azure raw-bytes handlers and dead palm completion code by @devin-ai-integration[bot] in #41662
refactor(vertex_ai): remove constant-False is_using_v1beta1_features stub and its dead call sites by @devin-ai-integration[bot] in #41665
fix(passthrough): keep target URL query when client sends no query params by @devin-ai-integration[bot] in #41448
fix(responses): drop top_p for gpt-5 reasoning models when drop_params is set by @devin-ai-integration[bot] in #41469
fix(ocr): keep a downloaded document inlined when callbacks intercept the request (#41550 follow-up) by @yuneng-berri in #41719
fix(models): rolling registry audit: Azure retirement dates, Bedrock Mantle Grok 4.3 context window by @devin-ai-integration[bot] in #41597
test(logging): add azure_spillover to the GCS pub/sub spend-log golden by @yuneng-berri in #41729
fix(proxy): forward every method on the typesafe pass-through route by @yuneng-berri in #41723

New Contributors

@max-sixty made their first contribution in #34829
@runjivu made their first contribution in #41379
@elifozdamar made their first contribution in #40843
@etiennechabert made their first contribution in #37983
@zachbernstein-sdx made their first contribution in #41444

Full Changelog: v1.103.0-dev.1...v1.103.0-dev.2