BerriAI/litellm v1.81.3.rc

on GitHub

What's Changed

• fix logfile and pidfile of supervisor for non root environment by @dotmobo in #17267
• Fix audio cost per second override by @dushyantzz in #19158
• fix: add openai/dall-e base pricing entries by @obod-mpw in #19133
• fix(tools): prevent OOM with nested $defs in tool schemas by @rsp2k in #19112
• chore: resolve ModuleNotFoundError for Microsoft Foundry Agents by @Harshit28j in #18991
• [Docs] Deleted Key and Team Table by @yuneng-jiang in #19299
• docs: add performance improvement section by @AlexsanderHamir in #19300
• [Infra] UI - Build Release UI Script by @yuneng-jiang in #19302
• Add in-product nudge for claude code feedback survey + new learning centre by @krrishdholakia in #19303
• [docs] add cursor with mcp by @uc4w6c in #19292
• Google WIF documentation by @krrishdholakia in #19320
• [fix] responses api non OpenAI models by @uc4w6c in #19317
• [test] mcp integration test by @uc4w6c in #19319
• Fix: _handle_failure method getting called 2 times by @Sameerlite in #19326
• Fix: upload pdfs for file endpoint by @Sameerlite in #19327
• Add managed files support when load_balancing is True by @Sameerlite in #19338
• Fix: anthropic-beta is getting overriden and set to anthropic-beta by @Sameerlite in #19343
• Fix Output None for replicate handler by @Sameerlite in #19347
• Fix: update the doc by @Sameerlite in #19349
• fix(router): prevent retrying 4xx client errors by @Harshit28j in #19275
• fix(agentcore): simplify agentcore streaming by @busla in #17141
• Revert "Fix audio cost per second override" by @Sameerlite in #19352
• fix(proxy): add /a2a/{agent_id}/.well-known/agent-card.json to agent_… by @Harshit28j in #19277
• Add ChatGPT subscription support and responses bridge by @msexxeta in #19030
• Fix: vector store sync issues by @Sameerlite in #19329
• fix(responses): streaming with tool_choice allowed_tools by @Chesars in #17205
• fix(logging): prevent duplicate StandardLoggingPayload logs by @Harshit28j in #19325
• fix(langfuse_otel): ignore service logs and fix callback shadowing by @Harshit28j in #19298
• feat: Add Redis-based migration lock with bug fixes by @Harshit28j in #19261
• fix(utils.py): correctly extract messages from google genai contents by @Harshit28j in #19156
• Litellm staging 01 16 2026 by @krrishdholakia in #19234
• [Fix] Bedrock stability model usage issues by @Sameerlite in #19323
• fix/bedrock-inconsistent-postcall-hook by @Harshit28j in #19151
• docs: update UI contributing guide by @Chesars in #19353
• Fix HTML entity in survey description text by @superpoussin22 in #19307
• Fix : test_responses_streaming_failure_triggers_failure_handlers by @Sameerlite in #19354
• Deployment limits note by @krrishdholakia in #19367
• Add health check scripts and parallel execution support by @AlexsanderHamir in #19295
• [Fix] Claude Code x Bedrock Invoke fails with advanced-tool-use-2025-11-20 by @ishaan-jaff in #19373
• feat: add retry_delay, exponential_backoff, and jitter to completion() by @Harshit28j in #19371
• docs: fix bad examples from sdk by @Harshit28j in #19322
• fix(realtime): disable SSL for ws:// WebSocket connections by @Chesars in #19345
• feat: add support for keda in helm chart by @rsicart in #19337
• fix: correct Groq gpt-oss pricing and add cache pricing by @cluebbehusen in #19311
• feat(azure): add support for Azure OpenAI v1 API by @Chesars in #19313
• feat(gemini): use responseJsonSchema for Gemini 2.0+ models by @Chesars in #19314
• fix(gcs_bucket): prevent unbounded queue growth due to slow API calls by @AlexsanderHamir in #19297
• fix #19254 - [Bug]: litellm_params ignored by get_llm_provider function in completion() definition by @Lucky-Lodhi2004 in #19315
• fix(bedrock): deduplicate tool calls in assistant history (#15178) by @binbandit in #19324
• [Feat] - Add self hosted Claude Code Plugin Marketplace by @ishaan-jaff in #19378
• Simplify comments on helper file by @AlexsanderHamir in #19382
• [Fix] Fix Pass through routes to work with server root path by @ishaan-jaff in #19383
• Fix #19357 - [Bug]: Tool call fails when using Ollama backend by @Lucky-Lodhi2004 in #19369
• fix: correct us.anthropic.claude-opus-4-5 In-region pricing by @cluebbehusen in #19310
• Fix queue persistence to Redis by @emerzon in #19304
• feat (volcengine) : Support Volcengine responses api by @LingXuanYin in #18508
• fix: HTTP client memory leaks in Presidio, OpenAI, and Gemini by @rsp2k in #19190
• [Feat] UI - Allow Adding Claude Code Plugins by @ishaan-jaff in #19387
• Fix: bedrock invoke claude 4 optional params #19318 by @flex-myeonghyeon in #19381
• docs: fix vertex_region to vertex_location in Vertex AI pass-through docs by @wwwillchen in #19380
• Field-Existence Checks to Type Classes to Prevent Attribute Errors by @Lrakotoson in #18321
• [feat] mcp version up by @uc4w6c in #19379
• fix(bedrock): handle thinking with tool calls for Claude 4 models by @bensi94 in #18787
• fix(responses): stream tool call events in completion bridge by @victorigualada in #19368
• docs: Migrate Pillar Security to Generic Guardrail API by @eagle-p in #19364
• fix: preserve tool output ordering for gemini in responses bridge by @victorigualada in #19360
• Fix extract_cacheable_prefix to handle string content with message-level cache_control by @VedantMadane in #19266
• fix: drop_params not dropping prompt_cache_key for non-OpenAI providers by @Chesars in #19346
• merge main iin 19/1 staging by @Sameerlite in #19400
• Merge main 01 20 2026 by @Sameerlite in #19401
• Revert "feat: add retry_delay, exponential_backoff, and jitter to completion()" by @Sameerlite in #19402
• Litellm staging 01 20 2026 fix by @Sameerlite in #19404
• Revert "fix #19254 - [Bug]: litellm_params ignored by get_llm_provider function in completion() definition" by @Sameerlite in #19405
• Revert "feat: Add Redis-based migration lock with bug fixes" by @Sameerlite in #19409
• Litellm staging 01 20 2026 by @krrishdholakia in #19386
• Add input_cost_per_video_per_second in ModelInfoBase by @Sameerlite in #19398
• Add gemini-2.5-computer-use-preview-10-2025 model for vertex ai provider by @Sameerlite in #19397
• Fix for Prometheus Metric Cardinality Issue with /responses Endpoint by @Sameerlite in #19396
• Litellm staging 01 19 2026 by @krrishdholakia in #19355
• Fix: ID mismatch between text-start and text-delta by @Sameerlite in #19390
• fix: remove count tokens optional param before request is sent to vertex by @otaviofbrito in #19359
• Fix/prisma schema permission by @Harshit28j in #19391
• fix(proxy_server): pass search_tools to Router during DB-triggered in… by @Harshit28j in #19388
• Fix: Handle PostgreSQL cached plan errors during rolling deployments by @AlexsanderHamir in #19424
• Supports setting media_resolution and fps parameters on each video file, when using Gemini video understanding. by @krisxia0506 in #19273
• Fix in-flight request termination on SIGTERM when health-check runs in a separate process by @AlexsanderHamir in #19427
• fix: UI dashboard respects custom authentication header override by @stiyyagura0901 in #19276
• fix(prompts): fix prompt info lookup and delete using correct IDs by @Harshit28j in #19358
• [Infra] UI - Fixing UI Build by @yuneng-jiang in #19431
• [Internal Tooling] Add custom auth header support and increase default prompt size to 100k chars by @AlexsanderHamir in #19436
• fix: sync Helm chart version with LiteLLM release version by @Chesars in #19438
• [Feature] UI - Playground: Button to Fill Custom API Base by @yuneng-jiang in #19440
• fix: add google-cloud-aiplatform as optional dependency with clear error message by @Chesars in #19437
• docs: clarify Gemini and Vertex AI model prefix in json file by @Chesars in #19443
• [Infra] Fixing LiteLLM Proxy Extras by @yuneng-jiang in #19446
• [Infra[ Fixing dynamic_router_retry_policy CI by @yuneng-jiang in https://github.com//pull/19448
• [Infra] Use mock db for claude code marketplace tests by @yuneng-jiang in #19451
• [Infra] Fix test_route_checks by @yuneng-jiang in #19450
• [Infra] Increase Time to Wait for Spend Accuracy Tests by @yuneng-jiang in #19452
• [Infra] Changing Google Tests to use Gemini 3 Flash Preview by @yuneng-jiang in #19456
• [Feat] Add Support for Claude Code Max/OAuth 2 on LiteLLM AI Gateway by @ishaan-jaff in #19453
• [Fix] Claude Code - /messages/token_counter - ensure it works for Anthropic, Azure AI Anthropic on AI Gateway by @ishaan-jaff in #19432
• Revert "[Infra] Changing Google Tests to use Gemini 3 Flash Preview" by @yuneng-jiang in #19457
• bump: version 1.81.0 → 1.81.1 by @yuneng-jiang in #19460
• fix: ensure auto-rotation updates existing AWS secret instead of crea… by @Harshit28j in #19455
• fix: resolve Read-only file system error in non-root images by @Harshit28j in #19449
• Make grpc dependency optional by @kamilio in #19447
• add search provider for brave search api by @jonathansampson in #19433
• fix: add better error handling for misconfig on health check by @Harshit28j in #19441
• fix(vertex_ai): handle reasoning_effort as dict from OpenAI Agents SDK by @Chesars in #19419
• fix #19414 - [Bug]: get_model_info on Bedrock suffixed models doesn't return proper information by @Lucky-Lodhi2004 in #19421
• fix(ui): increase model selector width in playground Compare view by @Chesars in #19423
• Fixes test_aaabasic_gcs_logger by @Sameerlite in #19463
• feat(gemini): add file content support in tool results by @rynecarbone in #19416
• Litellm cc docs max by @ishaan-jaff in #19466
• merge main 20 1 25 by @Sameerlite in #19491
• fix: correct gemini-2.5-flash-lite audio input and cache read pricing by @cluebbehusen in #19489
• Revert "Fix/prisma schema permission" by @Sameerlite in #19493
• Litellm staging 01 20 2026 fixes by @Sameerlite in #19495
• Litellm staging 01 20 2026 by @krrishdholakia in #19461
• Fix: [Bug]: litellm.litellm_core_utils.litellm_logging.Logging incorrectly handles ResponseAPIResponse by @Sameerlite in #19486
• merge main by @Sameerlite in #19497
• Filter out unsupported fields from JSON schema for Anthropic's output_format API. by @Sameerlite in #19482
• Add support for sarvam models by @Sameerlite in #19479
• Fix: tool call streaming in chat completion bridge by @Sameerlite in #19472
• Fix: total timeout is not respected by @Sameerlite in #19389
• Fix duplicate test_handler.py filenames causing pytest collection errors (#19330) by @jgreek in #19385
• [Feature] Paginate /v2/models/info by @yuneng-jiang in #19521
• [Fix] LiteLLM VertexAI Pass through - ensuring incoming headers are forwarded down to target by @ishaan-jaff in #19524
• [Feature] UI - Model Page: All Model Tab Pagination by @yuneng-jiang in #19525
• [feat] mcp spendlogs by @uc4w6c in #19469
• [fix] Send litellm_trace_id to Langfuse by @uc4w6c in #19528
• [Fix] UI - Virtual Keys Table: Sorting Shows Incorrect Entries by @yuneng-jiang in #19534
• feat: add GMI Cloud provider support by @Chesars in #19376
• Cut chat_completion latency by ~21% by reducing pre-call processing time by @AlexsanderHamir in #19535
• [Fix] UI - MCP Servers: MCP Tools Tab Resetting to Overview by @yuneng-jiang in #19468
• [Reissue: Fix] /user/new Privilege Escalation by @yuneng-jiang in #19296
• [Infra] UI - E2E Test: Create Key With All Team Models by @yuneng-jiang in #18995
• [Feature] Adding Optional scope Param to /models by @yuneng-jiang in #19539
• [Infra] UI - E2E Tests: Internal Viewer Sidebar by @yuneng-jiang in #19540
• [Fix] VertexAI Pass through - Ensure only anthropic betas are forwarded down to LLM API by @ishaan-jaff in #19542
• fix(mcp): forward static_headers to MCP servers (#19341) by @jayy-77 in #19366
• Add gpt-audio and gpt-audio-mini models to pricing by @Chesars in #19509
• fix(pricing): correct audio token costs for gpt-4o-audio-preview models by @Chesars in #19500
• docs(custom_pricing): add base_model usage for OpenAI dated model versions by @Chesars in #19502
• fix(azure): preserve content_policy_violation details for images (#19328) by @jayy-77 in #19372
• [Feat] Add Structured output for /v1/messages with Anthropic API, Azure Anthropic API, Bedrock Converse by @ishaan-jaff in #19545
• fixing prompt-security's guardrail implementation by @davida-ps in #19374
• Fix/per service ssl override v2 by @Harshit28j in #19538
• docs: update Claude Code integration guides by @Chesars in #19415
• fix: add missing redisvl dependency to requirements.txt by @houdataali in #19417
• [Fix] UI Cost Estimator - Fix model dropdown by @ishaan-jaff in #19529
• fix: UI 404 error when SERVER_ROOT_PATH is set by @Harshit28j in #19467
• fix: add case-insensitive support for guardrail mode and actions by @Harshit28j in #19480
• fix(bedrock): correct streaming choice index for tool calls by @joaodinissf in #19506
• Fix Azure RPM calculation formula by @emerzon in #19513
• Fix(Azure): Support OpenAI-format nested tool definitions for Responses API by @yogeshwaran10 in #19526
• Fix date overflow/division by zero in proxy utils by @emerzon in #19527
• Fix Azure AI costs for Anthropic models by @wwwillchen in #19530
• [Feature] UI - Create Team: Reusable Model Select by @yuneng-jiang in #19543
• [feat] Add MCP tools response to chat completions by @uc4w6c in #19552
• [feat] display mcp output on the play ground by @uc4w6c in #19553
• Litellm staging 01 21 2026 fixes by @Sameerlite in #19565
• Litellm staging 01 21 2026 by @krrishdholakia in #19464
• Fix: generation config empty for batch by @Sameerlite in #19556
• Add custom vertex ai mapping to the output by @Sameerlite in #19558
• [Feat] Add support for output formatfor bedrock invoke via v1/messages by @Sameerlite in #19560
• feat: Limit stop sequence as per openai spec by @Sameerlite in #19562
• Metrics prometheus user team count by @ecao310 in #19520
• [Infra] CI/CD - Fix UI Build + MyPy Linting by @yuneng-jiang in #19576
• [Infra] CI/CD - Adding python3-dev to Non Root Dockerfile by @yuneng-jiang in #19579
• [Infra] CI/CD - Adding node-tar CVE to Allowlist by @yuneng-jiang in #19581
• fix(websearch_interception): filter internal kwargs before follow-up request by @mpcusack-altos in #19577
• [Infra] CI/CD - Skip Brave Tests by @yuneng-jiang in #19582
• Fix unsafe access to request attribute by @AlexsanderHamir in #19573
• build(deps): bump lodash from 4.17.21 to 4.17.23 in /docs/my-website by @dependabot[bot] in #19570
• [Infra] CI/CD - Updating Prometheus Tests by @yuneng-jiang in #19583
• [Infra] CI/CD - Fix Non Root Proxy Tests by @yuneng-jiang in #19585
• [Infra] CI/CD - Adding lodash-es to allowlist by @yuneng-jiang in #19587
• Revert "[Infra] UI - E2E Tests: Internal Viewer Sidebar" by @yuneng-jiang in #19591
• [Infra] CI/CD - Fix LLM Translation Tests by @yuneng-jiang in #19590
• [Infra] CI/CD - Overriding lodash-es with v4.17.23 in docs and UI by @yuneng-jiang in #19593
• bump: version 1.81.1 → 1.81.2 by @yuneng-jiang in #19595
• [Infra] CI/CD - Add retries to Flaky Tests + Add "eos" as a Finish Reason by @yuneng-jiang in #19603
• add opencode tutorial under ai tools section by @milan-berri in #19602
• adjust opencode tutorial by @milan-berri in #19605
• feat(helm): Enable PreStop hook configuration in values.yaml by @Harshit28j in #19613
• [Feat] New LiteLLM Policy engine - create policies to manage guardrails, conditions - permissions per Key, Team by @ishaan-jaff in #19612
• fix: pass through endpoints update registry by @Harshit28j in #19420
• [Fix] Anthropic models on Azure AI cache pricing (#19532) by @jgreek in #19614
• [fix] completions mcp output ordering by @uc4w6c in #19623
• [test] Skip anthropic model test when ANTHROPIC_API_KEY is not set by @uc4w6c in #19624
• merge main in stagin 1 22 26 by @Sameerlite in #19638
• [fix] include tool arguments in proxy_server_request for spend logs ca… by @uc4w6c in #19640
• [feat] hashicorp vault rotate support by @uc4w6c in #19634
• Fix: Responses API logging error for StopIteration by @Sameerlite in #19649
• Add tool choice mapping for giga chat by @Sameerlite in #19645
• Litellm staging 01 22 2026 by @krrishdholakia in #19548
• fix #19620: SSO user roles are not updated for existing users by @xqe2011 in #19621
• [Infra] CI/CD - Linting, Security, Flaky Tests by @yuneng-jiang in #19662
• [Feat] Guardrail Policy Management - Allow using UI to manage guardrail policies by @ishaan-jaff in #19668
• fix: add openinference span kinds to arize phoenix by @mubashir1osmani in #19267
• add spend-queue-troubleshooting docs by @milan-berri in #19659
• [Feature] UI - Fallbacks: New Add Fallbacks Modal by @yuneng-jiang in #19673
• Add Langfuse mock mode for testing without API calls by @AlexsanderHamir in #19676
• Add GCS mock mode for testing without API calls by @AlexsanderHamir in #19683
• perf: Optimize strip_trailing_slash with O(1) index check by @ryan-crabbe in #19679
• perf: Optimize use_custom_pricing_for_model with set intersection by @ryan-crabbe in #19677
• perf: skip pattern_router.route() for non-wildcard models by @ryan-crabbe in #19664
• [Feature] UI - Create Team and Key Router Settings by @yuneng-jiang in #19686
• perf: Add LRU caching to get_model_info for faster cost lookups by @ryan-crabbe in #19606
• fix: ensure guardrail patterns persist on edit and mode toggle by @naaa760 in #19265
• [Fix] UI - Redirect to ui/login on expired JWT by @yuneng-jiang in #19687
• [Feat] UI + Backend - Allow adding policies on Keys/Teams + Viewing on Info panels by @ishaan-jaff in #19688
• docs: add litellm-enterprise requirement for managed files by @Harshit28j in #19689
• Update Gemini 2.0 Flash deprecation dates to March 31, 2026 by @Chesars in #19592
• [Infra] CI/CD - Fixing UI Build by @yuneng-jiang in #19694
• [Infra] CI/CD - Fixing failing tests by @yuneng-jiang in #19695
• [Infra] Fixing CircleCI Config by @yuneng-jiang in #19701
• [Feature] UI - Models Page: Model Search by @yuneng-jiang in #19622
• [Fix] Team Update with Organization having All Proxy Models by @yuneng-jiang in #19604
• [Feature] UI - Organization Edit Page: Reusable Model Select by @yuneng-jiang in #19601
• [Infra] Build UI for Release by @yuneng-jiang in #19703
• Fix: log duplication when json_logs is enabled by @AlexsanderHamir in #19705
• [Feature] UI - Model Page: Filter by Model ID and Team ID by @yuneng-jiang in #19713
• fix(presidio): resolve runtime error by handling asyncio loops in bac… by @Harshit28j in #19714
• [Docs] UI Keys Teams Router Settings docs by @yuneng-jiang in #19715
• [Infra] Rebuilding UI by @yuneng-jiang in #19716

New Contributors

• @dushyantzz made their first contribution in #19158
• @obod-mpw made their first contribution in #19133
• @msexxeta made their first contribution in #19030
• @rsicart made their first contribution in #19337
• @cluebbehusen made their first contribution in #19311
• @Lucky-Lodhi2004 made their first contribution in #19315
• @binbandit made their first contribution in #19324
• @flex-myeonghyeon made their first contribution in #19381
• @Lrakotoson made their first contribution in #18321
• @bensi94 made their first contribution in #18787
• @victorigualada made their first contribution in #19368
• @VedantMadane made their first contribution in #19266
• @stiyyagura0901 made their first contribution in #19276
• @kamilio made their first contribution in #19447
• @jonathansampson made their first contribution in #19433
• @rynecarbone made their first contribution in #19416
• @jayy-77 made their first contribution in #19366
• @davida-ps made their first contribution in #19374
• @joaodinissf made their first contribution in #19506
• @ecao310 made their first contribution in #19520
• @mpcusack-altos made their first contribution in #19577
• @milan-berri made their first contribution in #19602
• @xqe2011 made their first contribution in #19621

Full Changelog: v1.81.0-stable...v1.81.3.rc