BerriAI/litellm v1.102.0-dev.2 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.102.0-dev.2

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.102.0-dev.2/cosign.pub \
  ghcr.io/berriai/litellm:v1.102.0-dev.2

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

fix(mcp): preserve proxy logging and authorization coverage by @tin-berri in #40337
fix(spend-tracking): recover key alias for session tokens from spend logs by @devin-ai-integration[bot] in #40275
fix(stream_chunk_builder): guard empty choices and missing role in build_base_response by @ZXT-zjbiliy in #37781
fix(mcp): log proxy tool dispatch exceptions by @tin-berri in #40351
fix(azure_ai): price seven Foundry catalog names and charge the model router fee once by @devin-ai-integration[bot] in #40189
test(e2e/ui): cover team-scoped model visibility, re-editing litellm params, and model health checks by @yuneng-berri in #40039
test(e2e/ui): cover key budget window, non-admin model scope edit, and key blocking by @yuneng-berri in #40027
fix(mcp): honor an explicit null on toolset update, cover MCP lifecycle e2e by @yuneng-berri in #40022
fix(batches): clean up E2E resources across providers by @yuneng-berri in #40161
fix(model-management): honor an explicit null as a clear on model update by @yuneng-berri in #40047
feat(guardrails): map each guardrail scan id to its guardrail, stage and provider by @devin-ai-integration[bot] in #40327
revert(model-management): roll back #40047 by @yuneng-berri in #40383
fix(proxy): apply team model aliases on the JWT auth path by @ryan-crabbe-berri in #39985
fix(integrations): pass original request object to post-call guardrail hooks by @yujonglee-berri in #40414
fix(cost-map): keep first fetch blocking, run retries in background by @devin-ai-integration[bot] in #40350
fix(ui): repair pass-through delete confirm dialog and disable delete for config endpoints by @mubashir1osmani in #40303
feat(proxy): price cache and reasoning tokens in /cost/estimate by @ryan-crabbe-berri in #40174
fix(mcp): surface connection failures across transports by @joshua-berri in #40359
perf(proxy): pipeline spend counter increments into one Redis call per request by @devin-ai-integration[bot] in #40371
fix(proxy): ignore team_id="" on /key/update so team-less keys can be updated and imported by @devin-ai-integration[bot] in #40421
feat(router): support percentile-based TTFT routing by @clement-paradex in #40352
perf(proxy): collapse per-worker SGR upserts into one statement per flush by @devin-ai-integration[bot] in #40362
fix(router): strip encrypted reasoning on an auto-router tier change instead of a 503 by @tin-berri in #40280
test: isolate bedrock aws tests from ambient SSL env vars by @CaptainAni187 in #40425
fix(proxy): pin multi-root CA bundle to the server's root before handing it to Prisma by @devin-ai-integration[bot] in #40428
fix(cli): skip remote model cost map fetch in lite CLI processes by @devin-ai-integration[bot] in #40372
chore(lint): bring ANN202 and BLE001 back under the strict-rule budget by @devin-ai-integration[bot] in #40444
feat(guardrails): run post_call policy pipelines on background Responses retrieval by @devin-ai-integration[bot] in #40274
feat(auth): add disable_env_credential_login setting with admin ui warning by @ojensen-berri in #40116
test(ui): derive reasoning-effort assertion from the anthropic preset by @cursor[bot] in #40456
feat(hosted_vllm): add image edit support by @devin-ai-integration[bot] in #40329
fix(proxy): keep a body litellm_session_id in SpendLogs under missing_session_id omit by @devin-ai-integration[bot] in #40379
fix(rate_limiter): attach v3 priority rate limit headers on /v1/messages by @devin-ai-integration[bot] in #37228
feat(guardrails): deliver tool-call rewrites into buffered chat, Responses, and Messages streams by @devin-ai-integration[bot] in #40271
fix(proxy): accept non-string callback vars in default_team_settings by @devin-ai-integration[bot] in #40458
fix(router): resolve team-scoped auto-routers by their public name by @tin-berri in #40432
fix(langfuse): give each call in a session header its own trace instead of upserting one trace per session by @yucheng-berri in #40177
fix(databricks): keep top-level reasoning_content from OpenAI-compatible gateway models by @devin-ai-integration[bot] in #40449
fix(convert_dict_to_response): handle empty choices list without raising 500 APIError by @shotsan in #40294
fix(router): resolve route candidate ids through the router's own resolver by @tin-berri in #40491
fix(jwt): cascade-delete JWT key mappings when their virtual key is deleted by @devin-ai-integration[bot] in #33703
fix(databricks): route Unity model services through AI Gateway by @tin-berri in #40492
fix(ui): preserve dotted MCP tool argument names by @joshua-berri in #40494
feat(cli): add lite configure claude and lite unconfigure claude by @tin-berri in #40319
fix(ui): scope shadow eval models to configured chat groups by @tin-berri in #40488
fix(pricing): rolling model registry update: Bedrock gpt-6-astra, gpt-image-2.5, Cohere rerank 4, Vertex Grok 4.3/4.6/4.20, Gemini 3.5 audio, OpenAI web search fee, xAI Imagine video, Lyria 3.5, Voyage, ChatGPT GPT-5.5/5.6, Bedrock Mantle, Scaleway dates by @devin-ai-integration[bot] in #31884
fix(guardrails): scan and rewrite Responses custom_tool_call output items by @devin-ai-integration[bot] in #40461
feat(guardrails): run legacy post-call hooks as streaming pipeline steps by @devin-ai-integration[bot] in #40284
fix(azure): respect DEFAULT_MAX_RETRIES in initialize_azure_sdk_client (internal copy of #35771) by @devin-ai-integration[bot] in #40464
fix: keep reasoning_effort for mode: responses bridge deployments by @devin-ai-integration[bot] in #40249
fix(ui): guard playground cost metric against null and NaN by @devin-ai-integration[bot] in #40257
fix(responses): echo a named tool_choice in the Responses API shape on the chat-completions bridge by @devin-ai-integration[bot] in #40462
fix(azure_ai): reject unsupported n and size params on MAI image models by @mihidumh in #40074
fix(bedrock): stop leaking Anthropic thinking/reasoning_effort into DeepSeek Converse requests by @devin-ai-integration[bot] in #33409
test(e2e): every spend row a virtual key writes joins its token across all write paths by @devin-ai-integration[bot] in #40465
fix(token_counter): release the GIL for HuggingFace counts and cap exact counting per string by @devin-ai-integration[bot] in #40186
fix(bedrock): sign requests off the event loop on every async path by @devin-ai-integration[bot] in #40270
fix(cli): let the apiKeyHelper supply Claude Code's key under lite claude by @devin-ai-integration[bot] in #40489
fix(responses): keep background polling alive after the client disconnects by @devin-ai-integration[bot] in #40114
fix(terraform): preserve undeclared server-side key metadata on update by @devin-ai-integration[bot] in #40514
fix(terraform): read team per-model rpm/tpm limits from metadata and clear them on removal by @devin-ai-integration[bot] in #40439
feat(terraform): allow custom team_id on litellm_team by @devin-ai-integration[bot] in #40459
fix(anthropic): replay OpenAI encrypted reasoning byte for byte through the /v1/messages bridge by @devin-ai-integration[bot] in #40451
fix(terraform): send litellm_key model_max_budget as per-model BudgetConfig objects by @devin-ai-integration[bot] in #40450
fix(mcp): refresh tool previews when editing connection settings by @joshua-berri in #40498
fix(terraform): keep prior state when a virtual key update is rejected by @devin-ai-integration[bot] in #40512
fix(terraform): send changed litellm_key duration on /key/update so expires is recomputed by @devin-ai-integration[bot] in #40511
fix(terraform): drop litellm_key from state on 404 instead of failing the plan by @devin-ai-integration[bot] in #40443
fix(terraform): read key fields the proxy stores in metadata back from /key/info by @devin-ai-integration[bot] in #40513
chore(ci): promote internal staging to main by @devin-ai-integration[bot] in #40526
chore(ui): bump smol-toml to fix GHSA-7w5x-hrqm-74c2 osv-scan failure by @devin-ai-integration[bot] in #40497
fix(azure_ai): add passthrough config so router-model relays reach the deployment's own endpoint by @mateo-berri in #39863
fix(mcp): report resolved upstream authentication in debug headers by @joshua-berri in #40454
fix(mcp): challenge and scope gateway-owned server authentication by @joshua-berri in #40453
fix(mcp): respect optional discovery capabilities and quiet unsupported methods by @joshua-berri in #40525
fix(proxy): surface runtime-registered callbacks in UI Logging page by @yucheng-berri in #38974
feat(proxy): list gateway models to Claude Code under Claude-shaped ids and serve requests on them by @tin-berri in #40515
fix(openai): drop tool schema regex patterns OpenAI's validator cannot compile by @tin-berri in #40485
fix(proxy): bound concurrent key and spend-counter DB lookups to stop prisma pool thrash by @devin-ai-integration[bot] in #40387
fix(s3_v2): freeze refreshable credentials before signing and retry 403 uploads with a fresh signature by @devin-ai-integration[bot] in #40187
feat(prometheus): bucket latency by input sequence length by @yucheng-berri in #40059
fix(model_prices): registry audit 2026-09-10, absorb open pricing PRs by @devin-ai-integration[bot] in #40581
feat(bedrock): thread aws_session_tags into STS AssumeRole by @ryan-crabbe-berri in #40446
ci(lint): gate top-level tests/e2e and litellm files in the diff-scoped lint steps by @mateo-berri in #39272
feat(pointfive): add the pointfive logging integration by @yinonkahta-p5 in #38509
fix(dashscope): remap chat-shaped api_base to the live rerank route by @mateo-berri in #39237
fix(terraform): restore d.Partial(true) on a rejected /key/update by @devin-ai-integration[bot] in #40527
feat(mock): report admission-time input token count in mock_response usage by @devin-ai-integration[bot] in #40590
perf(proxy): reuse cached model group and deployment info in budget reservation by @devin-ai-integration[bot] in #40593
fix(router): strip Codex harness envelopes before classification by @moe-berri in #40599
feat(ocr): add core foundation and Mistral adapter by @yujonglee-berri in #40530
feat(infra): scale gateway on per-pod RPS and TPS in Helm and Terraform by @devin-ai-integration[bot] in #40479
feat(terraform): prometheus metrics sidecar for the GCP Cloud Run gateway by @devin-ai-integration[bot] in #40614
fix(proxy): recreate the Prisma client when the writer session turns read-only by @devin-ai-integration[bot] in #40610
feat(rust_bridge): count budget-check input tokens in Rust on all LLM routes by @devin-ai-integration[bot] in #40381
fix(wandb): preserve reasoning_effort in chat completions by @WolframRavenwolf in #39190
fix(router): classify encrypted delegated tasks with native Responses by @moe-berri in #40608
fix(mcp): reject initialize with 403 when the key grants no MCP servers by @devin-ai-integration[bot] in #40616
feat(claude-code): accept https zip archive plugin sources for skills by @devin-ai-integration[bot] in #40496
feat(proxy): granular key/team access control for Claude Code marketplace plugins by @devin-ai-integration[bot] in #40518
fix(voyage): accept flat list[str] input for contextual embeddings by @fzowl in #35091
fix(mcp): bind per-user OAuth credentials to the authenticated LiteLLM caller by @devin-ai-integration[bot] in #38724
fix(proxy): dedup latest health checks in SQL and gate the DB save per window by @michelligabriele in #39539
fix(proxy): resolve /v1/models limits from the deployment, not the alias by @michelligabriele in #39296
feat(redis): add ElastiCache IAM authentication by @eugene-yao-zocdoc in #38413
test(proxy): pass the request to get_marketplace in the archive marketplace test by @mateo-berri in #40626
feat(proxy): share database connections across workers with an in-container pgbouncer by @devin-ai-integration[bot] in #39683
fix(proxy): let internal users read request/response for their own spend logs by @krth1k in #35448
chore(github): disable blank issues so filers must use a template by @ryan-crabbe-berri in #40629
feat(router): log exact classifier input and masked source request by @moe-berri in #40604
fix(redis): log an open circuit breaker once instead of a traceback per request and count sync timeouts as timeouts by @mateo-berri in #40620
feat(proxy): let the in-container pgbouncer follow rotating RDS IAM and Azure Entra tokens by @devin-ai-integration[bot] in #40623
fix(proxy): log rejected unknown-model requests under a placeholder model name by @mateo-berri in #40622
feat(wandb): default unmapped W&B models to reasoning-capable by @ryan-crabbe-berri in #40625
test(ocr): isolate native Rust bridge contracts by @yujonglee-berri in #40410
feat(proxy): gate organization endpoints on an enterprise license by @ryan-crabbe-berri in #40613
feat(proxy): offload spend tracking to a pod-local collector sidecar by @devin-ai-integration[bot] in #40545
feat(newrelic): export team max and remaining budget gauges to the Metric API by @devin-ai-integration[bot] in #40542
fix(reset_budget_job): reset end users by budget link, not by user id by @ryan-crabbe-berri in #40639
perf(mock): emit admission-time usage chunk on streaming mock_response by @devin-ai-integration[bot] in #40637
fix(rate_limiter): skip non-Latin-1 x-litellm-priority header on /v1/messages by @devin-ai-integration[bot] in #40636
test: fix stale completion response fixtures by @yuneng-berri in #40638
fix(logging): finish response metadata before the sync logging thread reads it by @mateo-berri in #39869
feat(ui): link the Team, Organization, User and Created By cells on the Virtual Keys page by @ryan-crabbe-berri in #40646
test: respect optional logging payload fields by @yuneng-berri in #40643
fix(streaming): keep admitted mock streams alive with empty stream_options and honor zero prompt counts by @devin-ai-integration[bot] in #40650
chore: bump litellm-proxy-extras 0.4.95 -> 0.4.96 by @yuneng-berri in #40630
fix(helm): give the collector sidecar the pod PgBouncer env when database.connectionPool is enabled by @devin-ai-integration[bot] in #40660
feat(ui): link the entity cells on the team detail page's keys table by @ryan-crabbe-berri in #40647
fix(auto-router): omit Claude Code system text from classifier by @tin-berri in #40655
chore(ci): promote internal staging to main by @yuneng-berri in #40658

New Contributors

@ZXT-zjbiliy made their first contribution in #37781
@joshua-berri made their first contribution in #40359
@CaptainAni187 made their first contribution in #40425
@shotsan made their first contribution in #40294
@yinonkahta-p5 made their first contribution in #38509
@krth1k made their first contribution in #35448

Full Changelog: v1.102.0-dev.1...v1.102.0-dev.2