BerriAI/litellm v1.102.0 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.102.0

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.102.0/cosign.pub \
  ghcr.io/berriai/litellm:v1.102.0

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

feat(policy_engine): execute post_call guardrail pipelines on streaming responses by @mateo-berri in #38788
fix(policy_engine): apply post_call pipeline text rewrites on streams by @mateo-berri in #39233
fix(proxy): redact provider keys from pass-through failure tracebacks by @mateo-berri in #39964
fix(file_search): scope emulated file_search to the request's vector stores by @mateo-berri in #39972
feat(cli): add lite pi to run the pi coding agent through the proxy by @ryan-crabbe-berri in #36841
feat(pricing): add GovCloud pricing for every live but unpriced Bedrock model by @mateo-berri in #39764
fix(azure_sentinel): split batches under the 1MB ingestion cap by @yucheng-berri in #39880
feat(guardrails): add inspect_embeddings toggle for AIM and Cato by @yucheng-berri in #39918
refactor(ui): route the sidebar by pathname and shrink the ?page= shim to a redirect table by @ryan-crabbe-berri in #39978
fix(anthropic): keep provider_specific_fields off the native /v1/messages wire by @tin-berri in #39967
fix(headroom): inject headroom_retrieve only for service-declared ccr_hashes and keep assistant content blocks intact by @tin-berri in #39974
fix(auto_router): build the semantic route layer off the event loop by @moe-berri in #39954
fix(adaptive_router): fall back to model_info for cost-weighted scoring by @moe-berri in #39957
fix(adaptive_router): add the persisted delta to the cold-start prior on load by @moe-berri in #39955
feat(mcp): add opt-in per-server oauth relay discovery by @tin-berri in #39936
refactor(ui): render the Virtual Keys page without the legacy user dashboard by @ryan-crabbe-berri in #39991
feat(router): gate heuristic v1 tuning by @tin-berri in #39952
fix(docker): ship pymongo in the proxy images for the MongoDB vector store by @yuneng-berri in #39995
fix(router): treat a routing entry with no latency samples as zero latency by @mateo-berri in #39970
ci(e2e): run a PR's changed e2e tests three times behind a human-approved environment by @yuneng-berri in #39416
feat(vertex): add Lyria model support by @emerzon in #30856
ci: run unit tests on Python 3.12 by @yujonglee-berri in #39989
fix(anthropic_responses): preserve Responses refusal blocks in Anthropic messages translation by @Atharva-Kanherkar in #39723
fix(batches): account a batch's cost once, from the first retrieve that sees it final by @mateo-berri in #39980
feat(cost-map): add azure_ai/gpt-6-astra Foundry pricing by @mateo-berri in #39983
chore(lint): stop ratcheting *-budget.json on PR branches by @devin-ai-integration[bot] in #39937
ci: simplify Rust checks and remove wheel PR comments by @yujonglee-berri in #39975
fix(router): skip the refusing deployment when retrying a non-transient error by @mateo-berri in #40014
feat(mistral): add text-to-speech support for /v1/audio/speech by @mateo-berri in #38755
fix(mcp): apply key and team guardrails to MCP tool calls by @mubashir1osmani in #39629
test(ui): pin wire contracts for key, model and MCP server forms by @yuneng-berri in #40019
refactor(rust): remove per-request enablement arguments by @yujonglee-berri in #39928
test(ocr): trace callback lifecycle parity by @yujonglee-berri in #40063
fix(router): serialize heuristic tuning quota filters for Prisma by @yuneng-berri in #40140
docs(claude): have runs embed their own QA screenshots on visual changes by @mateo-berri in #39800
fix(oci): stream Cohere tool-calling answers once by @mateo-berri in #39965
fix(async): move remote image fetches off the event loop for Snowflake, Bedrock invoke Claude, Mantle and Gemini by @mateo-berri in #39839
fix(ai-gateway): dial upstream WebSockets over an explicit rustls provider by @mateo-berri in #39530
fix(cost): bill realtime reasoning tokens nested in text_tokens once by @mateo-berri in #39850
fix: keep provider id and metadata on Responses API bridged chat completions by @mateo-berri in #39981
fix(make check): lint the test tree on tests-only changes like CI does by @mateo-berri in #39870
fix(ci): stop the auto-close duplicates job colliding with the required test check by @mateo-berri in #40015
fix: capture provider request id in failure logging payloads by @yucheng-berri in #40045
test(ui): make navigation smoke resilient to router refactors by @yuneng-berri in #40026
test(e2e/ui): cover member role and budget edits, member permission delegation, and team guardrail removal by @yuneng-berri in #40042
test(ocr): add SDK callback E2E parity by @yujonglee-berri in #40061
test(e2e): prove the virtual key lifecycle on every gateway replica by @yuneng-berri in #40023
fix(spend_logs): keep partition DDL transactions alive for their statement timeout by @devin-ai-integration[bot] in #40098
feat(proxy): resolve root_path per request from a configured prefix list (SERVER_ROOT_PATHS) by @gym-cmd in #35935
fix(router): preserve heuristic defaults across releases by @tin-berri in #40007
fix(proxy): enforce key and team model access on Bedrock passthrough routes (internal copy of #34244) by @mateo-berri in #39660
feat(ci): add the cost map guard check by @mateo-berri in #39835
feat(ui): add key-scoped auto-router usage tab by @tin-berri in #39999
fix(ui): hide the Create Vector Store flow from non proxy admins by @devin-ai-integration[bot] in #40148
fix(ui): send empty vector_stores when the last team vector store is removed by @devin-ai-integration[bot] in #40144
feat(skills): semantic search over the LiteLLM-hosted skill registry by @yassin-berriai in #39401
fix(ui): show inherited MCP servers on the internal user editor and flag access groups with no members by @devin-ai-integration[bot] in #40036
feat(keys): allow editing soft budget on existing keys by @devin-ai-integration[bot] in #39002
fix(guardrails): block Prompt Security file modifications by @davida-ps in #38204
fix(router): keep provider response headers on streaming chat completions by @yucheng-berri in #40091
ci: prepare workflows for main default branch by @yuneng-berri in #40149
fix(proxy): let authorized internal users open vector store details by @devin-ai-integration[bot] in #40150
feat(ui): make automatic auto-router setup discoverable and show what it configured by @tin-berri in #40146
fix(docs): fix stale file paths in ARCHITECTURE.md by @jrlprost in #40157
fix(guardrails): accept on_violation block and alert for mcp_security by @devin-ai-integration[bot] in #40155
fix(spend): price caching savings on the billed request basis by @tin-berri in #40160
fix(responses): drop unsupported reasoning param for openai non-reasoning models by @mateo-berri in #38842
fix(guardrails): resolve generateContent routes and async-first passthrough call types by @mateo-berri in #38869
fix(mcp): forward staged credentials on /mcp-rest/test/connection like /test/tools/list by @mateo-berri in #38806
fix(proxy): register SkillsInjectionHook at proxy startup instead of import time by @mateo-berri in #38914
feat(ui): list the ChatGPT subscription provider in the Add Model form by @ryan-crabbe-berri in #40170
test(guardrails): pin the presidio spend-log record and the UI's masked-entity persistence by @yuneng-berri in #40038
test(e2e/ui): automate the RC checklist's Presidio guardrail walk by @yuneng-berri in #40041
ci: follow the default branch in development tooling by @yuneng-berri in #40162
fix(mcp): encrypt stored static headers and stdio environment by @tin-berri in #40164
feat(fireworks_ai): add native Responses API config by @mateo-berri in #39826
feat(ui): itemize auto-router classification spend by @tin-berri in #40168
fix(bedrock): keep x-amzn-RequestId on chat error responses by @yucheng-berri in #40089
fix(mcp): clear error when an agent-bound key is denied a scoped MCP server + agent MCP grants in the UI by @mateo-berri in #39234
fix(anthropic): skip one-shot Claude Code cache injection by @tin-berri in #40175
feat(complexity_router): add declarative custom dimensions to the heuristic scorer by @tin-berri in #40156
fix(proxy): log budget reservation notice once at config load by @yucheng-berri in #40167
test: add Rust extension pytest contract by @yujonglee-berri in #40181
test: drop tests that only pin static cost-map values by @devin-ai-integration[bot] in #40195
fix(vertex_ai): support fine-tuned Gemini endpoints in managed batches by @mubashir1osmani in #39668
fix(spend): compare auto-router targets by deployment identity by @tin-berri in #40206
feat(ui): show auto-router classification rate by @tin-berri in #40192
feat(mcp): start the named server's OAuth directly for a resource-scoped gateway flow by @tin-berri in #39933
fix(router): rank streaming latency routing by raw TTFT, not TTFT per token by @clement-paradex in #40202
fix(tool_permission): log expected skip and deny events below WARNING by @devin-ai-integration[bot] in #40208
fix(proxy): kill the whole prisma process group when a boot migration command times out by @mateo-berri in #39509
fix(proxy): load db credentials in the model reconcile so a worker never serves a model before its credential by @mateo-berri in #39876
fix(router): give cooldowns their own cache so siblings see a bench in ~1s by @mateo-berri in #40025
fix(least-busy): share in-flight request counts across proxy workers by @mateo-berri in #40009
fix(proxy-extras): rebuild indexes left INVALID by a migration deadlock by @mateo-berri in #39384
fix(guardrails): keep guardrail telemetry when a policy pipeline blocks or modifies the response by @devin-ai-integration[bot] in #40211
fix(ocr): run post-call logging hooks by @yujonglee-berri in #40154
fix(guardrails): allow framework-supported logging-only mode by @yujonglee-berri in #40267
fix(responses bridge): keep mid-conversation system messages in input instead of folding them into instructions by @devin-ai-integration[bot] in #40269
fix(proxy): initialize string success/failure callbacks at startup after config load by @devin-ai-integration[bot] in #38226
fix(responses): record spend for native Responses API WebSocket sessions by @devin-ai-integration[bot] in #38856
feat(complexity_router): rebalance heuristic weights in the dashboard and grade custom dimensions by match count by @tin-berri in #40205
feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform by @devin-ai-integration[bot] in #40163
feat(router): resolve max_tokens to the tier model's ceiling on auto-routed requests by @tin-berri in #40209
chore: bump litellm-proxy-extras 0.4.94 -> 0.4.95, litellm 1.101.0 -> 1.102.0 by @yuneng-berri in #40287
feat(otel): make the OTel v2 trace export URL configurable by @devin-ai-integration[bot] in #40286
fix(router): keep per-request routing_strategy override selectors out of global callbacks by @devin-ai-integration[bot] in #40229
fix(router): count allowed_fails in the shared router cache so multi-worker proxies bench a deployment fleet-wide by @devin-ai-integration[bot] in #40224
fix(router): let simple-shuffle weight by any deployment's weight/rpm/tpm by @devin-ai-integration[bot] in #40222
fix(router): give cost-based routing its own cache key so it stops overwriting latency samples by @devin-ai-integration[bot] in #40225
fix(router): keep deployment tags out of retry and fallback tag routing by @devin-ai-integration[bot] in #40226
fix(proxy): keep team member budget enforced at the cap and across Redis counter expiry by @devin-ai-integration[bot] in #40304
fix(utils): honor string drop_params values from config and DB deployments by @mateo-berri in #33738
fix(policy_engine): execute post_call guardrail pipelines on responses and streams by @mateo-berri in #38721
feat(ui): batch observability on the logs page by @mubashir1osmani in #39626
feat(otel): add http/json export protocol for OTel v2 traces by @devin-ai-integration[bot] in #40290
feat: move MongoDB vector search to an optional sidecar (BETA) by @yuneng-berri in #40203
perf: move Anthropic, Vertex Anthropic, Ollama and HF template fetches off the event loop by @devin-ai-integration[bot] in #40311
fix(router): move retry-policy retries off the refusing deployment on every router entrypoint by @devin-ai-integration[bot] in #40306
fix(proxy): default max_idle_connection_lifetime on componentized DB URLs by @devin-ai-integration[bot] in #40285
fix(a2a): forward caller identity headers on message/send and message/stream by @yassin-berriai in #40305
feat(team): let a team admin manage their own team's logging callbacks by @yucheng-berri in #37667
fix(proxy): give each spend-log queue monitor its own flush event by @mateo-berri in #39556
fix(mlflow): prevent _stream_id_to_span leak and mlflow 2.x end_trace TypeError by @devin-ai-integration[bot] in #39049
feat(otel v2): send a key's or team's whole trace to its own destination by @yucheng-berri in #39654
chore(ui): update dashboard dependencies by @yuneng-berri in #40312
feat(bedrock): add TwelveLabs Marengo Embed 3.0 embeddings by @devin-ai-integration[bot] in #40180
fix(proxy): stop shipping the literal string "None" as error type and param by @mateo-berri in #39536
chore(ci): merge main into internal staging by @yuneng-berri in #40323
test(azure_sentinel): pin batch_size as a per-request bound under concurrent events by @devin-ai-integration[bot] in #40320
fix(fireworks_ai): fold instructions and developer items into one leading system message on the Responses path by @devin-ai-integration[bot] in #40268
feat(auto_router): opt-in NON_REASONING tier below SIMPLE by @moe-berri in #40273
fix(budget_reservation): don't reserve budget on token counting routes by @devin-ai-integration[bot] in #36718
feat(mcp): add schema discovery proxy mode by @tin-berri in #40298
test(proxy): fix Prisma timeout cleanup after subreaper tests by @yuneng-berri in #40333
fix(otel v2): restore the Datadog auth span and the last-wins callback merge by @devin-ai-integration[bot] in #40335
chore(ci): extend diskcache scan exception to October 1 by @yuneng-berri in #40336
fix(anthropic): key the /v1/messages prompt cache on Claude Code's session_id only by @ryan-crabbe-berri in #40342
feat(auto-router): refresh family reasoning presets by @tin-berri in #40341
feat(cost_map): report which revision of the price map the proxy is serving by @devin-ai-integration[bot] in #40179
test(mcp): fix proxy fixture isolation after manager reload by @yuneng-berri in #40347
chore: bump litellm-enterprise 0.1.65 -> 0.1.66 by @yuneng-berri in #40334
chore(ci): promote internal staging to main by @yuneng-berri in #40307
fix(mcp): preserve proxy logging and authorization coverage by @tin-berri in #40337
fix(spend-tracking): recover key alias for session tokens from spend logs by @devin-ai-integration[bot] in #40275
fix(stream_chunk_builder): guard empty choices and missing role in build_base_response by @ZXT-zjbiliy in #37781
fix(mcp): log proxy tool dispatch exceptions by @tin-berri in #40351
fix(azure_ai): price seven Foundry catalog names and charge the model router fee once by @devin-ai-integration[bot] in #40189
test(e2e/ui): cover team-scoped model visibility, re-editing litellm params, and model health checks by @yuneng-berri in #40039
test(e2e/ui): cover key budget window, non-admin model scope edit, and key blocking by @yuneng-berri in #40027
fix(mcp): honor an explicit null on toolset update, cover MCP lifecycle e2e by @yuneng-berri in #40022
fix(batches): clean up E2E resources across providers by @yuneng-berri in #40161
fix(model-management): honor an explicit null as a clear on model update by @yuneng-berri in #40047
feat(guardrails): map each guardrail scan id to its guardrail, stage and provider by @devin-ai-integration[bot] in #40327
revert(model-management): roll back #40047 by @yuneng-berri in #40383
fix(proxy): apply team model aliases on the JWT auth path by @ryan-crabbe-berri in #39985
fix(integrations): pass original request object to post-call guardrail hooks by @yujonglee-berri in #40414
fix(cost-map): keep first fetch blocking, run retries in background by @devin-ai-integration[bot] in #40350
fix(ui): repair pass-through delete confirm dialog and disable delete for config endpoints by @mubashir1osmani in #40303
feat(proxy): price cache and reasoning tokens in /cost/estimate by @ryan-crabbe-berri in #40174
fix(mcp): surface connection failures across transports by @joshua-berri in #40359
perf(proxy): pipeline spend counter increments into one Redis call per request by @devin-ai-integration[bot] in #40371
fix(proxy): ignore team_id="" on /key/update so team-less keys can be updated and imported by @devin-ai-integration[bot] in #40421
feat(router): support percentile-based TTFT routing by @clement-paradex in #40352
perf(proxy): collapse per-worker SGR upserts into one statement per flush by @devin-ai-integration[bot] in #40362
fix(router): strip encrypted reasoning on an auto-router tier change instead of a 503 by @tin-berri in #40280
test: isolate bedrock aws tests from ambient SSL env vars by @CaptainAni187 in #40425
fix(proxy): pin multi-root CA bundle to the server's root before handing it to Prisma by @devin-ai-integration[bot] in #40428
fix(cli): skip remote model cost map fetch in lite CLI processes by @devin-ai-integration[bot] in #40372
chore(lint): bring ANN202 and BLE001 back under the strict-rule budget by @devin-ai-integration[bot] in #40444
feat(guardrails): run post_call policy pipelines on background Responses retrieval by @devin-ai-integration[bot] in #40274
feat(auth): add disable_env_credential_login setting with admin ui warning by @ojensen-berri in #40116
test(ui): derive reasoning-effort assertion from the anthropic preset by @cursor[bot] in #40456
feat(hosted_vllm): add image edit support by @devin-ai-integration[bot] in #40329
fix(proxy): keep a body litellm_session_id in SpendLogs under missing_session_id omit by @devin-ai-integration[bot] in #40379
fix(rate_limiter): attach v3 priority rate limit headers on /v1/messages by @devin-ai-integration[bot] in #37228
feat(guardrails): deliver tool-call rewrites into buffered chat, Responses, and Messages streams by @devin-ai-integration[bot] in #40271
fix(proxy): accept non-string callback vars in default_team_settings by @devin-ai-integration[bot] in #40458
fix(router): resolve team-scoped auto-routers by their public name by @tin-berri in #40432
fix(langfuse): give each call in a session header its own trace instead of upserting one trace per session by @yucheng-berri in #40177
fix(databricks): keep top-level reasoning_content from OpenAI-compatible gateway models by @devin-ai-integration[bot] in #40449
fix(convert_dict_to_response): handle empty choices list without raising 500 APIError by @shotsan in #40294
fix(router): resolve route candidate ids through the router's own resolver by @tin-berri in #40491
fix(jwt): cascade-delete JWT key mappings when their virtual key is deleted by @devin-ai-integration[bot] in #33703
fix(databricks): route Unity model services through AI Gateway by @tin-berri in #40492
fix(ui): preserve dotted MCP tool argument names by @joshua-berri in #40494
feat(cli): add lite configure claude and lite unconfigure claude by @tin-berri in #40319
fix(ui): scope shadow eval models to configured chat groups by @tin-berri in #40488
fix(pricing): rolling model registry update: Bedrock gpt-6-astra, gpt-image-2.5, Cohere rerank 4, Vertex Grok 4.3/4.6/4.20, Gemini 3.5 audio, OpenAI web search fee, xAI Imagine video, Lyria 3.5, Voyage, ChatGPT GPT-5.5/5.6, Bedrock Mantle, Scaleway dates by @devin-ai-integration[bot] in #31884
fix(guardrails): scan and rewrite Responses custom_tool_call output items by @devin-ai-integration[bot] in #40461
feat(guardrails): run legacy post-call hooks as streaming pipeline steps by @devin-ai-integration[bot] in #40284
fix(azure): respect DEFAULT_MAX_RETRIES in initialize_azure_sdk_client (internal copy of #35771) by @devin-ai-integration[bot] in #40464
fix: keep reasoning_effort for mode: responses bridge deployments by @devin-ai-integration[bot] in #40249
fix(ui): guard playground cost metric against null and NaN by @devin-ai-integration[bot] in #40257
fix(responses): echo a named tool_choice in the Responses API shape on the chat-completions bridge by @devin-ai-integration[bot] in #40462
fix(azure_ai): reject unsupported n and size params on MAI image models by @mihidumh in #40074
fix(bedrock): stop leaking Anthropic thinking/reasoning_effort into DeepSeek Converse requests by @devin-ai-integration[bot] in #33409
test(e2e): every spend row a virtual key writes joins its token across all write paths by @devin-ai-integration[bot] in #40465
fix(token_counter): release the GIL for HuggingFace counts and cap exact counting per string by @devin-ai-integration[bot] in #40186
fix(bedrock): sign requests off the event loop on every async path by @devin-ai-integration[bot] in #40270
fix(cli): let the apiKeyHelper supply Claude Code's key under lite claude by @devin-ai-integration[bot] in #40489
fix(responses): keep background polling alive after the client disconnects by @devin-ai-integration[bot] in #40114
fix(terraform): preserve undeclared server-side key metadata on update by @devin-ai-integration[bot] in #40514
fix(terraform): read team per-model rpm/tpm limits from metadata and clear them on removal by @devin-ai-integration[bot] in #40439
feat(terraform): allow custom team_id on litellm_team by @devin-ai-integration[bot] in #40459
fix(anthropic): replay OpenAI encrypted reasoning byte for byte through the /v1/messages bridge by @devin-ai-integration[bot] in #40451
fix(terraform): send litellm_key model_max_budget as per-model BudgetConfig objects by @devin-ai-integration[bot] in #40450
fix(mcp): refresh tool previews when editing connection settings by @joshua-berri in #40498
fix(terraform): keep prior state when a virtual key update is rejected by @devin-ai-integration[bot] in #40512
fix(terraform): send changed litellm_key duration on /key/update so expires is recomputed by @devin-ai-integration[bot] in #40511
fix(terraform): drop litellm_key from state on 404 instead of failing the plan by @devin-ai-integration[bot] in #40443
fix(terraform): read key fields the proxy stores in metadata back from /key/info by @devin-ai-integration[bot] in #40513
chore(ci): promote internal staging to main by @devin-ai-integration[bot] in #40526
chore(ui): bump smol-toml to fix GHSA-7w5x-hrqm-74c2 osv-scan failure by @devin-ai-integration[bot] in #40497
fix(azure_ai): add passthrough config so router-model relays reach the deployment's own endpoint by @mateo-berri in #39863
fix(mcp): report resolved upstream authentication in debug headers by @joshua-berri in #40454
fix(mcp): challenge and scope gateway-owned server authentication by @joshua-berri in #40453
fix(mcp): respect optional discovery capabilities and quiet unsupported methods by @joshua-berri in #40525
fix(proxy): surface runtime-registered callbacks in UI Logging page by @yucheng-berri in #38974
feat(proxy): list gateway models to Claude Code under Claude-shaped ids and serve requests on them by @tin-berri in #40515
fix(openai): drop tool schema regex patterns OpenAI's validator cannot compile by @tin-berri in #40485
fix(proxy): bound concurrent key and spend-counter DB lookups to stop prisma pool thrash by @devin-ai-integration[bot] in #40387
fix(s3_v2): freeze refreshable credentials before signing and retry 403 uploads with a fresh signature by @devin-ai-integration[bot] in #40187
feat(prometheus): bucket latency by input sequence length by @yucheng-berri in #40059
fix(model_prices): registry audit 2026-09-10, absorb open pricing PRs by @devin-ai-integration[bot] in #40581
feat(bedrock): thread aws_session_tags into STS AssumeRole by @ryan-crabbe-berri in #40446
ci(lint): gate top-level tests/e2e and litellm files in the diff-scoped lint steps by @mateo-berri in #39272
feat(pointfive): add the pointfive logging integration by @yinonkahta-p5 in #38509
fix(dashscope): remap chat-shaped api_base to the live rerank route by @mateo-berri in #39237
fix(terraform): restore d.Partial(true) on a rejected /key/update by @devin-ai-integration[bot] in #40527
feat(mock): report admission-time input token count in mock_response usage by @devin-ai-integration[bot] in #40590
perf(proxy): reuse cached model group and deployment info in budget reservation by @devin-ai-integration[bot] in #40593
fix(router): strip Codex harness envelopes before classification by @moe-berri in #40599
feat(ocr): add core foundation and Mistral adapter by @yujonglee-berri in #40530
feat(infra): scale gateway on per-pod RPS and TPS in Helm and Terraform by @devin-ai-integration[bot] in #40479
feat(terraform): prometheus metrics sidecar for the GCP Cloud Run gateway by @devin-ai-integration[bot] in #40614
fix(proxy): recreate the Prisma client when the writer session turns read-only by @devin-ai-integration[bot] in #40610
feat(rust_bridge): count budget-check input tokens in Rust on all LLM routes by @devin-ai-integration[bot] in #40381
fix(wandb): preserve reasoning_effort in chat completions by @WolframRavenwolf in #39190
fix(router): classify encrypted delegated tasks with native Responses by @moe-berri in #40608
fix(mcp): reject initialize with 403 when the key grants no MCP servers by @devin-ai-integration[bot] in #40616
feat(claude-code): accept https zip archive plugin sources for skills by @devin-ai-integration[bot] in #40496
feat(proxy): granular key/team access control for Claude Code marketplace plugins by @devin-ai-integration[bot] in #40518
fix(voyage): accept flat list[str] input for contextual embeddings by @fzowl in #35091
fix(mcp): bind per-user OAuth credentials to the authenticated LiteLLM caller by @devin-ai-integration[bot] in #38724
fix(proxy): dedup latest health checks in SQL and gate the DB save per window by @michelligabriele in #39539
fix(proxy): resolve /v1/models limits from the deployment, not the alias by @michelligabriele in #39296
feat(redis): add ElastiCache IAM authentication by @eugene-yao-zocdoc in #38413
test(proxy): pass the request to get_marketplace in the archive marketplace test by @mateo-berri in #40626
feat(proxy): share database connections across workers with an in-container pgbouncer by @devin-ai-integration[bot] in #39683
fix(proxy): let internal users read request/response for their own spend logs by @krth1k in #35448
chore(github): disable blank issues so filers must use a template by @ryan-crabbe-berri in #40629
feat(router): log exact classifier input and masked source request by @moe-berri in #40604
fix(redis): log an open circuit breaker once instead of a traceback per request and count sync timeouts as timeouts by @mateo-berri in #40620
feat(proxy): let the in-container pgbouncer follow rotating RDS IAM and Azure Entra tokens by @devin-ai-integration[bot] in #40623
fix(proxy): log rejected unknown-model requests under a placeholder model name by @mateo-berri in #40622
feat(wandb): default unmapped W&B models to reasoning-capable by @ryan-crabbe-berri in #40625
test(ocr): isolate native Rust bridge contracts by @yujonglee-berri in #40410
feat(proxy): gate organization endpoints on an enterprise license by @ryan-crabbe-berri in #40613
feat(proxy): offload spend tracking to a pod-local collector sidecar by @devin-ai-integration[bot] in #40545
feat(newrelic): export team max and remaining budget gauges to the Metric API by @devin-ai-integration[bot] in #40542
fix(reset_budget_job): reset end users by budget link, not by user id by @ryan-crabbe-berri in #40639
perf(mock): emit admission-time usage chunk on streaming mock_response by @devin-ai-integration[bot] in #40637
fix(rate_limiter): skip non-Latin-1 x-litellm-priority header on /v1/messages by @devin-ai-integration[bot] in #40636
test: fix stale completion response fixtures by @yuneng-berri in #40638
fix(logging): finish response metadata before the sync logging thread reads it by @mateo-berri in #39869
feat(ui): link the Team, Organization, User and Created By cells on the Virtual Keys page by @ryan-crabbe-berri in #40646
test: respect optional logging payload fields by @yuneng-berri in #40643
fix(streaming): keep admitted mock streams alive with empty stream_options and honor zero prompt counts by @devin-ai-integration[bot] in #40650
chore: bump litellm-proxy-extras 0.4.95 -> 0.4.96 by @yuneng-berri in #40630
fix(helm): give the collector sidecar the pod PgBouncer env when database.connectionPool is enabled by @devin-ai-integration[bot] in #40660
feat(ui): link the entity cells on the team detail page's keys table by @ryan-crabbe-berri in #40647
fix(auto-router): omit Claude Code system text from classifier by @tin-berri in #40655
chore(ci): promote internal staging to main by @yuneng-berri in #40658
fix(caching): keep an open Redis circuit breaker open and quiet on the sync read and spend counter paths by @devin-ai-integration[bot] in #40624
fix(mcp): accept VS Code OAuth registration callbacks by @joshua-berri in #40664
fix(mcp): check OpenAPI specifications without native MCP handshakes by @joshua-berri in #40665
fix(model_prices): registry audit rolling PR: deepseek-flash, gpt-live-1, xAI/Groq deprecation dates, Perplexity Nemotron reasoning by @devin-ai-integration[bot] in #40606
perf(proxy): lazy-load provider passthrough routes by @devin-ai-integration[bot] in #40691
fix(proxy): give user-key objects their own in-memory cache partition by @devin-ai-integration[bot] in #40713
fix(passthrough): parse Bedrock stream spend incrementally instead of buffering the whole response by @devin-ai-integration[bot] in #40724
feat(secret_managers): support customer-managed KMS key for virtual keys stored in AWS Secrets Manager by @devin-ai-integration[bot] in #40475
feat(proxy): make the in-memory management cache capacity configurable by @devin-ai-integration[bot] in #40725
perf(proxy): register liveness and core inference routes first by @devin-ai-integration[bot] in #40687
docs(router): name both affinity TTL knobs in the _claim_pin docstring by @anmolg1997 in #40663
fix(mcp): explain refused OAuth registration and bound discovery retries by @joshua-berri in #40679
fix(mcp): log upstream request method, body and response on tool-list and OAuth2 token failures by @devin-ai-integration[bot] in #40440
fix(oci): pin one response id per streamed completion, skip the [DONE] sentinel by @mateo-berri in #39507
fix(vector-stores): surface retrieval failures to the API caller by @mateo-berri in #39516
fix(ai-gateway): build the release image again and cover it in CI by @mateo-berri in #39523
fix(proxy): authorize every Responses API id, not only the ones the proxy issued by @mateo-berri in #39548
fix(router): fall back from unhealthy auto-router tier by @devin-ai-integration[bot] in #40757
fix(proxy): keep call_type and request start time on failed-request spend logs by @devin-ai-integration[bot] in #40558
fix(proxy): retain metadata when retrieving public team aliases by @haydster7 in #40554
test(e2e): wait for serving propagation in UI journeys by @yuneng-berri in #40754
refactor(ocr): route native requests through core by @yujonglee-berri in #40532
feat(auto-router): show the routed model and session savings in Claude Code and Codex by @tin-berri in #40330
fix(mcp): write failure spend log for guardrail-blocked /mcp-rest/tools/call by @devin-ai-integration[bot] in #40555
feat(ocr): add Azure Mistral adapter and document fetching by @yujonglee-berri in #40533
fix(datadog_llm_obs): keep tool call and result structure under redaction and emit tool output tokens by @devin-ai-integration[bot] in #40666
fix(search): propagate GET provider HTTP errors by @yuneng-berri in #40779
fix(policy_engine): run global policy pipelines before scoped ones by @devin-ai-integration[bot] in #39697
fix(content_filter): log only scan time as streaming post_call guardrail duration by @devin-ai-integration[bot] in #40760
fix(cost-map): bedrock reasoning effort flags, registry audit fixes for vertex/openai/together/openrouter, absorb cerebras and inception rows by @devin-ai-integration[bot] in #40740
test(responses): fix stale Anthropic smoke request by @yuneng-berri in #40786
fix(ui): jump straight to the last Request Logs page instead of advancing one page by @devin-ai-integration[bot] in #40644
feat(ocr): add Azure Mistral adapter with native authentication by @yujonglee-berri in #40502
feat(ocr): add Azure Document Intelligence adapter by @yujonglee-berri in #40534
feat(ocr): add Reducto legacy and v3 adapters by @yujonglee-berri in #40535
feat(ocr): add Vertex Mistral adapter by @yujonglee-berri in #40507
feat(ocr): add Vertex DeepSeek adapter and remove legacy OCR pipeline by @yujonglee-berri in #40509
fix(vertex_ai): return 400 for invalid reasoning_effort instead of 500 by @ryan-crabbe-berri in #40748
fix(guardrails): stop logging the request payload as guardrail_response on pre_call hooks by @devin-ai-integration[bot] in #39699
feat(rust): count tiktoken cl100k_base admission tokens in Rust by @devin-ai-integration[bot] in #40777
fix(hide-secrets): restore credential coverage lost to the 4.5 entropy limit by @yucheng-berri in #40190
test(load): add a Redis timeout chaos load test by @kerry-berri in #40482
feat(proxy): expose complexity routing headers by @devin-ai-integration[bot] in #40792
fix(otel v2): name Langfuse traces from the langfuse_trace_name header or metadata.trace_name by @devin-ai-integration[bot] in #40793
feat(ui): link the Organization cell on the Teams page by @ryan-crabbe-berri in #40749
docs(pr-template): note untested assumptions under Caveats by @kerry-berri in #40811
feat(ui): link the User ID and Team ID cells on the Memory page by @ryan-crabbe-berri in #40752
feat(ui): link the User ID, Created By and Deleted By cells on Deleted Keys by @ryan-crabbe-berri in #40750
feat(ui): link the Created By cell on the Prompts page by @ryan-crabbe-berri in #40753
feat(ui): link the Organization and Deleted By cells on Deleted Teams by @ryan-crabbe-berri in #40751
fix(proxy): emit internal user budget webhook alerts by @jon-walton in #40396
fix(proxy): prevent spend counter double counting by @dclarksymmetry in #40572
fix(guardrails): keep post_call guardrail info on streamed chat completions by @devin-ai-integration[bot] in #40806
fix(realtime): dial Azure's GA realtime upstream for GA clients by @devin-ai-integration[bot] in #40769
fix(ui): preserve cleared shared select values by @yuneng-berri in #40795
feat(model_prices): add DeepSeek V4.1 Flash on Fireworks by @devin-ai-integration[bot] in #40812
fix(fireworks_ai): keep reasoning_content on replayed assistant messages by @devin-ai-integration[bot] in #40682
fix(shadow-eval): skip hosted web search samples by @devin-ai-integration[bot] in #40827
fix(db): carry DATABASE_SSLMODE/DATABASE_SSLROOTCERT into the assembled writer and reader URLs by @devin-ai-integration[bot] in #40815
fix(guardrails): fail closed with a named error when a Responses input rewrite cannot be applied by @csoni-cweave in #40609
fix(ui): make the env-credential login warning banner dismissible by @devin-ai-integration[bot] in #40831
test(pgbouncer): stop the never-listens replacement test flaking under CI load by @devin-ai-integration[bot] in #40830
feat(ui): search, sort and role filter for the team member table by @devin-ai-integration[bot] in #40659
fix(proxy): keep the raw model string out of the unknown-model spend-log error message by @devin-ai-integration[bot] in #40820
fix(mcp): cache upstream discovery lists by @joshua-berri in #40790
fix(mcp): match per-server OAuth metadata issuers by @joshua-berri in #40808
fix(mcp): use gateway authentication for root discovery by @joshua-berri in #40791
fix(bedrock_mantle): gate reasoning.summary on the OpenAI Responses path by @devin-ai-integration[bot] in #40798
test(e2e): memory regression test for failing requests on the release gate by @devin-ai-integration[bot] in #40773
test(e2e): reusable JWT fixtures and management lifecycle coverage by @ryan-crabbe-berri in #39990
fix(proxy): expand access groups in /health scoping and allowlist health display fields by @devin-ai-integration[bot] in #40765
test(ocr): exempt native parity requests from cassette replay by @yuneng-berri in #40838
fix(proxy): persist clearing user model budgets by @yuneng-berri in #40837
perf(auth): read user, team, membership, org, project and spend counters in one MGET, one query and one pipeline by @devin-ai-integration[bot] in #40834
perf(proxy): one MGET and one pipeline for post-call spend counters, no team/user/org refetch on the response path by @devin-ai-integration[bot] in #40841
fix(responses): preserve hosted web search calls by @devin-ai-integration[bot] in #40828
feat(cli): configure Claude Code and Codex with a gateway key by @tin-berri in #40829
fix(router): restore compression inheritance when clearing overrides by @yuneng-berri in #40839
fix(keys): support explicit project detachment by @yuneng-berri in #40836
feat(guardrails): add Conduct Guard integration with validated hooks and forwarded params by @devin-ai-integration[bot] in #40785
fix(mcp): enforce end user mcp_tool_permissions on tools/list and tools/call by @devin-ai-integration[bot] in #40865
refactor(ocr): complete native lifecycle and preserve Azure auth by @yujonglee-berri in #40734
fix(ui): restore MCP catalog provider logos by @yuneng-berri in #40781
fix(ui): preserve clear and default semantics in local forms by @yuneng-berri in #40826
fix(ui): clarify blank TPM/RPM hint on budget modals by @devin-ai-integration[bot] in #40697
test(e2e): verify cached answers and upstream request count by @yuneng-berri in #40774
fix(key): recover from a cascade-deleted key instead of failing the apply by @ryan-crabbe-berri in #40880
fix(proxy): accept both deferred stream logging arg shapes on native routes by @devin-ai-integration[bot] in #40869
perf(policy_engine): dedup attachments in one pass after sorting by @devin-ai-integration[bot] in #40883
fix(ui): persist cleared budgets and reset intervals by @yuneng-berri in #40895
fix(guardrails): log mask when a guardrail adds request keys by @devin-ai-integration[bot] in #40882
fix(ui): show loading state instead of stale rows while a table search is pending by @devin-ai-integration[bot] in #40656
chore: bump litellm-enterprise 0.1.66 -> 0.1.67, litellm-proxy-extras 0.4.96 -> 0.4.97 by @yuneng-berri in #40905
test(fireworks): stop pinning prices in the cost-map tests by @devin-ai-integration[bot] in #40901
feat(ui): search Key Activity by key alias, key hash, user id, or email by @devin-ai-integration[bot] in #40652
fix(proxy): run migrations through python -m prisma when the prisma console script is not on PATH by @devin-ai-integration[bot] in #40768
feat(proxy): serve registered skills as an Agent Skills well-known index by @devin-ai-integration[bot] in #40770
test: deflake redis semantic cache sys.modules leak, LangSmith init loop patch, wall-clock stagger assertion, and zombie grandchild check in the fake prisma cli by @devin-ai-integration[bot] in #39895
fix(anthropic): price recovered tokens when a /v1/messages client disconnects mid-stream by @devin-ai-integration[bot] in #40766
fix(proxy): resolve config include directives for bucket-hosted configs by @devin-ai-integration[bot] in #40772
fix(registry): sync Azure/Together deprecation dates, fix computer-use-preview and OpenRouter metadata by @devin-ai-integration[bot] in #40855
test: tighten regression tests added in #37974 by @devin-ai-integration[bot] in #40771
fix(redis): count pool wait timeouts as breaker timeouts by @devin-ai-integration[bot] in #40764
fix(cost): honour deployment custom pricing for OCR calls by @devin-ai-integration[bot] in #40767
fix(cost): honour deployment custom pricing for OCR calls by @mihidumh in #36609
feat(realtime): add Meta Muse Voice transcription by @seyeong-han in #39395
fix(proxy): attribute gate-rejected requests to their endpoint in cache analytics by @devin-ai-integration[bot] in #40824
feat(registry): add openai reasoning-family fallback generalization by @devin-ai-integration[bot] in #40902
fix(proxy): allow key_alias substring matching on /key/list for non-admins by @devin-ai-integration[bot] in #40907
fix(databricks): translate reasoning_effort to thinking for Gemini 2.5 by @devin-ai-integration[bot] in #40909
fix(databricks): translate reasoning_effort to thinking for Gemini 2.5 by @cohml in #32786
fix(model_management): stop persisting cost map pricing as a deployment override by @ryan-crabbe-berri in #36222
chore: rebuild Admin UI bundle from staging by @yuneng-berri in #40931
chore(ci): promote internal staging to main by @yuneng-berri in #40933
fix(responses): backport request-param leak fixes to rc/1.102.0 (#41018, #41141, #41144) by @yuneng-berri in #41342
fix(images): backport the image[] and mask[] form key drop to rc/1.102.0 (#39512) by @mateo-berri in #41697
fix(license): backport the wildcard license auto_router grant to rc/1.102.0 (#41684) by @mateo-berri in #41701
fix: backport eight backport-stable fixes to rc/1.102.0 (#40596, #41046, #41086, #41171, #41178, #41283, #41495, #41689) by @mateo-berri in #41854
fix(team): apply team_member_budget updates to members still on the team default (backport of #41347 to rc/1.102.0) by @mateo-berri in #41935
test(e2e): backport the Nova Sonic nova-2-sonic model fix to rc/1.102.0 by @yuneng-berri in #42059

New Contributors

@gym-cmd made their first contribution in #35935
@jrlprost made their first contribution in #40157
@clement-paradex made their first contribution in #40202
@ZXT-zjbiliy made their first contribution in #37781
@CaptainAni187 made their first contribution in #40425
@shotsan made their first contribution in #40294
@yinonkahta-p5 made their first contribution in #38509
@krth1k made their first contribution in #35448
@haydster7 made their first contribution in #40554
@jon-walton made their first contribution in #40396
@dclarksymmetry made their first contribution in #40572

Full Changelog: v1.101.0...v1.102.0