BerriAI/litellm v1.81.7.dev1 on GitHub

What's Changed

test(proxy): add regression tests for vertex passthrough model names … by @michelligabriele in #19855
fix: guardrails issues streaming-response regex by @Harshit28j in #19901
fix: add fix for migration issue and and stable image by @Harshit28j in #19843
fix: filter unsupported beta headers for AWS Bedrock Invoke API by @jayy-77 in #19877
fix: allow tool_choice for Azure GPT-5 chat models by @jayy-77 in #19813
fix: tool with antropic #19800 by @zifeo in #19805
inspect BadRequestError after all other policy types by @demoray in #19878
fix(main): use local tiktoken cache in lazy loading by @Chesars in #19774
fix(gemini): subtract implicit cached tokens from text_tokens for correct cost calculation by @Chesars in #19775
fix Prompt Studio history to load tools and system messages by @naaa760 in #19920
[Release Day] - Fixed CI/CD issues & changed processes by @AlexsanderHamir in #19902
[Feat] - Search API add /list endpoint to list what search tools exist in router by @ishaan-jaff in #19969
[Feature] UI - Tables: Reusable Table Sort Component by @yuneng-jiang in #19970
[Feature] UI - Logs: Adding Error message search to ui spend logs by @yuneng-jiang in #19963
[Feat] LiteLLM Vector Stores - Add permission management for users, teams by @ishaan-jaff in #19972
feat: Add new OpenRouter models: xiaomi/mimo-v2-flash, `z-ai/glm-4.… by @rushilchugh01 in #19938
fix gemini gemini-robotics-er-1.5-preview entry by @Sameerlite in #19974
fix(vertex_ai): convert image URLs to base64 in tool messages for Anthropic by @Chesars in #19896
Fix/router search tools v2 by @Harshit28j in #19840
[Infra] Remove _experimental/out routes from gitignore + UI Build by @yuneng-jiang in #19976
[Feature] UI - Usage Export: Breakdown by Teams and Keys by @yuneng-jiang in #19953
Fix stream_chunk_builder to preserve images from streaming chunks by @Chesars in #19654
fix(docker): add libsndfile to main Dockerfile for ARM64 audio processing by @Chesars in #19776
fix(proxy): add datadog_llm_observability to /health/services allowed… by @michelligabriele in #19952
fix(proxy): prevent provider-prefixed model leaks by @bcdonadio in #19943
fix(hosted_vllm): route through base_llm_http_handler to support ssl_verify by @cfchase in #19893
Add OpenRouter Kimi K2.5 by @ayim in #19872
Add test to check if model map is corretly formatted by @Sameerlite in #19992
Add validation of model cost map as job by @Sameerlite in #19993
Fix model map path in validation test by @Sameerlite in #19994
Fix: litellm_fix_robotic_model_map_entry by @Sameerlite in #19997
oss staging 01/28/2026 by @krrishdholakia in #19906
Add custom_llm_provider as gemini translation by @Sameerlite in #19988
[Fix] Sorting for /v2/model/info by @yuneng-jiang in #19971
[Feature] Bulk Update Keys Endpoint by @yuneng-jiang in #19886
[Fix] error_code in Spend Logs metadata by @yuneng-jiang in #20015
[Feature] UI - Spend Logs: Show Current Store and Retention Status by @yuneng-jiang in #20017
[Feature] UI - New Badge Dot Render by @yuneng-jiang in #20024
Add event-driven coordination for global spend query to prevent cache stampede by @AlexsanderHamir in #20030
[Feat] New Model - amazon.nova-2-pro-preview-20251202-v1:0 by @ishaan-jaff in #20033
[Feat] LiteLLM x Claude Agent SDK Integration by @ishaan-jaff in #20035
[Docs] Claude Agents SDK x LiteLLM Guide by @ishaan-jaff in #20036
fix: run prisma generate as nobody user in non-root container by @milan-berri in #20000
merge main in passthrough by @Sameerlite in #20042
Add /openai_passthrough route for openai passthrough requests: by @Sameerlite in #19989
fix(gemini): support file retrieval in GoogleAIStudioFilesHandle by @varunsripad123 in #20018
fix(ResponseAPILoggingUtils): extract input tokens details as dict by @nht1206 in #20046
Fix max_input_tokens for gpt-5.2-codex by @genga6 in #20009
Litellm oss staging 01 29 2026 by @krrishdholakia in #19975
feat: add /delete endpoint support for gemini by @Sameerlite in #20055
Fix: Batch and File user level permissions by @Sameerlite in #19981
[Feat]Add cost tracking and usage object in aretrieve_batch call type by @Sameerlite in #19986
Add routing of xai chat completions to responses when web search options is present by @Sameerlite in #20051
Add disable flag for anthropic gemini cache translation by @Sameerlite in #20052
fix aspectRatio mapping in image edit by @Sameerlite in #20053
Fix: vllm embedding format by @Sameerlite in #20056
Fix: remove unsupported prompt-caching-scope-2026-01-05 header for vertex ai by @Sameerlite in #20058
[Feature] UI - Usage: Model Breakdown Per Key by @yuneng-jiang in #20039
Add mock client factory pattern and mock support for PostHog, Helicone, and Braintrust integrations by @AlexsanderHamir in #19707
Realtime API benchmarks by @ishaan-jaff in #20074
fixes: ci pipeline router coverage failure by @Harshit28j in #20065
[cookbook] - add section for using claude agent sdk + MCPs with LiteLLM by @ishaan-jaff in #20081
[Feat] Add async_post_call_response_headers_hook to CustomLogger by @ishaan-jaff in #20083
fix(proxy): resolve high CPU when router_settings in DB by avoiding REGISTRY.collect() in PrometheusServicesLogger by @AlexsanderHamir in #20087
Revert logs view commits by @ishaan-jaff in #20090
[Fix] UI - Navbar: Fixed Default Logo + Bound Logo Box by @yuneng-jiang in #20092
[Refactor] UI - Navbar: User Dropdown by @yuneng-jiang in #20095
[Feat] v2 - Logs view with side panel and improved UX by @ishaan-jaff in #20091
[Feat] UI - New View to render "Tools" on Logs View by @ishaan-jaff in #20093
[Feat] UI - Add Pretty print view of request/response by @ishaan-jaff in #20096
fixed mcp tools instructions on ui to show comma seprated str instead… by @shivamrawat1 in #20101
litellm_fix: add missing timezone import to proxy_server.py by @shin-bot-litellm in #20121
litellm_fix(proxy): reduce PLR0915 complexity (minimal) by @shin-bot-litellm in #20127
litellm_fix(ui): remove unused ToolOutlined import by @shin-bot-litellm in #20129
litellm_fix(e2e): disable bedrock-converse-claude-sonnet-4.5 in tests by @shin-bot-litellm in #20131
litellm_fix(test): fix Azure AI cost calculator test - use Logging class by @shin-bot-litellm in #20134
litellm_fix(test): fix Bedrock tool search header test regression by @shin-bot-litellm in #20135
litellm_fix(test): allow comment field in schema and exclude robotics models from tpm check by @shin-bot-litellm in #20139
litellm_docs: add missing environment variable documentation by @shin-bot-litellm in #20138
litellm_fix(test): add acancel_batch to Azure SDK client initialization test by @shin-bot-litellm in #20143
litellm_fix: handle unknown models in Azure AI cost calculator by @shin-bot-litellm in #20150
litellm_fix(test): fix router silent experiment tests to properly mock async functions by @shin-bot-litellm in #20140
[Feature] UI - Dark Mode: Delete Resource Modal by @yuneng-jiang in #20098
[Fix] UI - Vector Store: Allow Config Defined Models to Be Selected by @yuneng-jiang in #20031
[Fix] Add WATSONX_ZENAPIKEY to WatsonX credentials by @yuneng-jiang in #20086
[Infa] UI Build by @yuneng-jiang in #20154
litellm_fix: use get_async_httpx_client for logo download by @shin-bot-litellm in #20155
litellm_fix: check for agent mode before requiring DD_API_KEY/DD_SITE by @shin-bot-litellm in #20156
litellm_fix: handle empty dict for web_search_options in Nova grounding by @shin-bot-litellm in #20159
litellm_mypy_fix_batch1: fix type errors in files, opentelemetry, gemini transformation, key management by @shin-bot-litellm in #20161
litellm_fix(test): update Prometheus metric test assertions with new labels by @shin-bot-litellm in #20162
litellm_fix: remove hosted_vllm from OpenAI client tests by @shin-bot-litellm in #20163
litellm_fix: bump litellm-proxy-extras version to 0.4.28 by @shin-bot-litellm in #20166
litellm_fix(mypy): fix remaining type errors by @shin-bot-litellm in #20164
litellm_fix(security): allowlist Next.js CVEs for 7 days by @shin-bot-litellm in #20169
litellm_fix(router): use safe_deep_copy in _get_silent_experiment_kwargs by @shin-bot-litellm in #20170
docs(embeddings): add supported input formats section by @Chesars in #20073
litellm_fix(lint): remove unused ToolNameValidationResult imports by @shin-bot-litellm in #20176
litellm_fix(azure): Fix acancel_batch not using Azure SDK client initialization by @shin-bot-litellm in #20168
[Fix] Model Name During Fallback by @yuneng-jiang in #20177
[Fix] Health Endpoints when Callback Objects Defined by @yuneng-jiang in #20182
fix(test): add router.acancel_batch coverage by @shin-bot-litellm in #20183
fix(mypy): fix validate_tool_name return type signatures by @shin-bot-litellm in #20184
litellm_fix_flaky_batch_completion_test by @shin-bot-litellm in #20186
litellm_fix_cost_calc_test: correct prompt_tokens in test_string_cost_values by @shin-bot-litellm in #20185
docs/blog index page by @ryan-crabbe in #20188
[Docs] UI Spend Logs Settings Docs by @yuneng-jiang in #20197
[Doc] Fixing Image by @yuneng-jiang in #20198
litellm_fix_mapped_tests_core: clear client cache and fix isinstance checks by @shin-bot-litellm in #20196
docs: fix dead links in v1.81.6 release notes by @shin-bot-litellm in #20218
litellm_fix_mapped_tests_core: fix test isolation and mock injection issues by @shin-bot-litellm in #20209
docs: Update v1.81.6 release notes - Logs v2 with Tool Call Tracing by @shin-bot-litellm in #20225
feat: Support dimensions param for Cohere embed v4 by @amirzaushnizer in #20235
feat: add User-Agent customization support by @jayy-77 in #19881
feat(bedrock): add 1hr tiered caching costs for long-context models (#18988) by @cscguochang in #20214
Update Vertex AI Text to Speech doc to show use of audio by @Sameerlite in #20255
fix: add reasoning param support for GPT OSS cerebras by @Sameerlite in #20258
Fix: Slack alert issue by @Sameerlite in #20257
fix: Map reasoning content to anthropic thinking block(streaming+non-streaming) by @Sameerlite in #20254
Fix open_ai_embedding_models to have custom_llm_provider None by @Sameerlite in #20253
[Feat] Add support for 0 cost models by @Sameerlite in #20249
Add Anthropic caching and context tests by @Sameerlite in #20247
Litellm test bedrock optional params embeddings dimension by @Sameerlite in #20262
[Feat]Add support for nova sonic Speech to speech model by @Sameerlite in #20244
Litellm oss staging 01 31 2026 3 by @Sameerlite in #20266
perf: optimize wrapper_async with CallTypes caching and reduced lookups by @ryan-crabbe in #20204
perf: cache _get_relevant_args_to_use_for_logging() at module level by @ryan-crabbe in #20077
docs: Add FAQ for setting up and verifying LITELLM_LICENSE by @shin-bot-litellm in #20284
Model request tags documentation by @ishaan-jaff in #20290
[Infra] UI - Update next to 16.1.6 by @yuneng-jiang in #20220
[Fix] Remove Key Blocking on Login by @yuneng-jiang in #20210
feat: add Kimi K2.5 model entries for Moonshot provider by @krauckbot in #20273
fix: MCP "Session not found" error on VSCode reconnect by @shin-bot-litellm in #20298
[Feature] SSO Config Team Mappings by @yuneng-jiang in #20111
feat: add base /scim/v2 endpoint for SCIM resource discovery by @shin-bot-litellm in #20301
docs: add Prisma migration troubleshooting guide by @shin-bot-litellm in #20300
[Fix] UI - Team Settings: Disable Global Guardrail Persistence by @yuneng-jiang in #20307
[Feature] Key reset_spend endpoint by @yuneng-jiang in #20305
[Feature] UI - SSO: Add Team Mappings by @yuneng-jiang in #20299
Add blog post: Achieving Sub-Millisecond Proxy Overhead by @AlexsanderHamir in #20309
[Feat] - MCP Semantic Filtering Support by @ishaan-jaff in #20296
Litellm docs mcp filtering semantic by @ishaan-jaff in #20316
feat(guardrails): implement team-based isolation guardrails mgmnt by @Harshit28j in #19889
fixes failure metrics labels by @Harshit28j in #20152
fix: proxy failure cases, now log ip and user agent, key hash, name by @Harshit28j in #20145
[Feat] /chat/completions - allow using OpenAI style tools for web_search with VertexAI/gemini models by @ishaan-jaff in #20280
Add documentation correctly for nova sonic by @Sameerlite in #20320
[Feature] UI - Default Team Settings: Migrate Default Team Settings to use Reusable Model Select by @yuneng-jiang in #20310
[Feature] UI - Navbar: Option to Hide Community Engagement Buttons by @yuneng-jiang in #20308
feat(sdk): add proxy_auth for auto OAuth2/JWT token management by @Chesars in #20238
fix(github_copilot): preserve system prompts and auto-add headers by @Chesars in #20113
fix: add missing capability flags to vercel_ai_gateway models by @krauckbot in #20276
Litellm tuesday cicd release by @Sameerlite in #20328
Revert "Litellm tuesday cicd release" by @Sameerlite in #20330
Litellm tuesday cicd release final by @Sameerlite in #20333
bump litellm 1.81.7 by @Sameerlite in #20336
update 02 staging PR by @Sameerlite in #20337
adding together ai models to litellm models json by @FelipeRodriguesGare in #20319
[Feat] Allow calling A2A agents through LiteLLM /chat/completions API by @ishaan-jaff in #20358
UI - Show team alias on Models health page by @ishaan-jaff in #20359
fix: check for model_response_choices before guardrail input by @agrattan0820 in #19784
Fix fail-open for grayswan and pass metadata to cygnal api endpoint by @Reapor-Yurnero in #19837
[Feat] Use A2A registered agents with /chat/completions by @ishaan-jaff in #20362
[Feature] UI - Keys: Allowed Routes to Key Info and Edit Pages by @yuneng-jiang in #20369
fix: revert httpx client caching that caused closed client errors by @michelligabriele in #20025
Chore: Antd Modal Deprecated Props by @yuneng-jiang in #20317
[Fix] /user/update Allow for max_budget Resets by @yuneng-jiang in #20375
[Feature] UI - User Budget Page: Unlimited Budget Checkbox by @yuneng-jiang in #20380
[Bug] Ensure MCP permissions are enforced when using JWT Auth by @ishaan-jaff in #20383
Add support for delete and GET via file_id for gemini by @Sameerlite in #20329
Fix: Extra inputs are not permitted, field: 'messages[2].provider_specific_fields by @Sameerlite in #20334
Fix: Managed Batches: Inconsistent State Management for list and cancel batches by @Sameerlite in #20331
fix(proxy): forward extra headers in chat by @naaa760 in #20386
Custom Code Guardrails UI Playground by @krrishdholakia in #20377
[Feat] Add xAI /realtime API Support - works with LiveKitSDK by @ishaan-jaff in #20381
[Feature] Include Config Defined Search Tools in /search_tools/list by @yuneng-jiang in #20371
Revert "feat(guardrails): implement team-based isolation guardrails mgmnt" by @krrishdholakia in #20393
Semantic filter test skip by @krrishdholakia in #20387
Add gemini deep research in model cost map by @Sameerlite in #20406
[DOCS]Add copilotkit sdk doc as supported agents sdk by @Sameerlite in #20396
Revert "fix: proxy failure cases, now log ip and user agent, key hash, name" by @Sameerlite in #20413
feat(guardrails): implement team-based isolation guardrails mgmnt (#1… by @krrishdholakia in #20318
Add Key info endpoint object permission data by @Sameerlite in #20407
Fix supports_native_streaming for gemini and vertex ai and claude models by @Sameerlite in #20408
Fix: empty assistant message for converse API by @Sameerlite in #20390
Add mapping for responses tools in file ids by @Sameerlite in #20402

New Contributors

@zifeo made their first contribution in #19805
@rushilchugh01 made their first contribution in #19938
@cfchase made their first contribution in #19893
@ayim made their first contribution in #19872
@varunsripad123 made their first contribution in #20018
@nht1206 made their first contribution in #20046
@genga6 made their first contribution in #20009
@shin-bot-litellm made their first contribution in #20121
@amirzaushnizer made their first contribution in #20235
@cscguochang made their first contribution in #20214
@krauckbot made their first contribution in #20273
@agrattan0820 made their first contribution in #19784

Full Changelog: v1.81.3.rc.5...v1.81.7.dev1