BerriAI/litellm v1.81.7-nightly

on GitHub

What's Changed

• test(proxy): add regression tests for vertex passthrough model names … by @michelligabriele in #19855
• fix: guardrails issues streaming-response regex by @Harshit28j in #19901
• fix: add fix for migration issue and and stable image by @Harshit28j in #19843
• fix: filter unsupported beta headers for AWS Bedrock Invoke API by @jayy-77 in #19877
• fix: allow tool_choice for Azure GPT-5 chat models by @jayy-77 in #19813
• fix: tool with antropic #19800 by @zifeo in #19805
• inspect BadRequestError after all other policy types by @demoray in #19878
• fix(main): use local tiktoken cache in lazy loading by @Chesars in #19774
• fix(gemini): subtract implicit cached tokens from text_tokens for correct cost calculation by @Chesars in #19775
• fix Prompt Studio history to load tools and system messages by @naaa760 in #19920
• [Release Day] - Fixed CI/CD issues & changed processes by @AlexsanderHamir in #19902
• [Feat] - Search API add /list endpoint to list what search tools exist in router by @ishaan-jaff in #19969
• [Feature] UI - Tables: Reusable Table Sort Component by @yuneng-jiang in #19970
• [Feature] UI - Logs: Adding Error message search to ui spend logs by @yuneng-jiang in #19963
• [Feat] LiteLLM Vector Stores - Add permission management for users, teams by @ishaan-jaff in #19972
• feat: Add new OpenRouter models: xiaomi/mimo-v2-flash, `z-ai/glm-4.… by @rushilchugh01 in #19938
• fix gemini gemini-robotics-er-1.5-preview entry by @Sameerlite in #19974
• fix(vertex_ai): convert image URLs to base64 in tool messages for Anthropic by @Chesars in #19896
• Fix/router search tools v2 by @Harshit28j in #19840
• [Infra] Remove _experimental/out routes from gitignore + UI Build by @yuneng-jiang in #19976
• [Feature] UI - Usage Export: Breakdown by Teams and Keys by @yuneng-jiang in #19953
• Fix stream_chunk_builder to preserve images from streaming chunks by @Chesars in #19654
• fix(docker): add libsndfile to main Dockerfile for ARM64 audio processing by @Chesars in #19776
• fix(proxy): add datadog_llm_observability to /health/services allowed… by @michelligabriele in #19952
• fix(proxy): prevent provider-prefixed model leaks by @bcdonadio in #19943
• fix(hosted_vllm): route through base_llm_http_handler to support ssl_verify by @cfchase in #19893
• Add OpenRouter Kimi K2.5 by @ayim in #19872
• Add test to check if model map is corretly formatted by @Sameerlite in #19992
• Add validation of model cost map as job by @Sameerlite in #19993
• Fix model map path in validation test by @Sameerlite in #19994
• Fix: litellm_fix_robotic_model_map_entry by @Sameerlite in #19997
• oss staging 01/28/2026 by @krrishdholakia in #19906
• Add custom_llm_provider as gemini translation by @Sameerlite in #19988
• [Fix] Sorting for /v2/model/info by @yuneng-jiang in #19971
• [Feature] Bulk Update Keys Endpoint by @yuneng-jiang in #19886
• [Fix] error_code in Spend Logs metadata by @yuneng-jiang in #20015
• [Feature] UI - Spend Logs: Show Current Store and Retention Status by @yuneng-jiang in #20017
• [Feature] UI - New Badge Dot Render by @yuneng-jiang in #20024
• Add event-driven coordination for global spend query to prevent cache stampede by @AlexsanderHamir in #20030
• [Feat] New Model - amazon.nova-2-pro-preview-20251202-v1:0 by @ishaan-jaff in #20033
• [Feat] LiteLLM x Claude Agent SDK Integration by @ishaan-jaff in #20035
• [Docs] Claude Agents SDK x LiteLLM Guide by @ishaan-jaff in #20036
• fix: run prisma generate as nobody user in non-root container by @milan-berri in #20000
• merge main in passthrough by @Sameerlite in #20042
• Add /openai_passthrough route for openai passthrough requests: by @Sameerlite in #19989
• fix(gemini): support file retrieval in GoogleAIStudioFilesHandle by @varunsripad123 in #20018
• fix(ResponseAPILoggingUtils): extract input tokens details as dict by @nht1206 in #20046
• Fix max_input_tokens for gpt-5.2-codex by @genga6 in #20009
• Litellm oss staging 01 29 2026 by @krrishdholakia in #19975
• feat: add /delete endpoint support for gemini by @Sameerlite in #20055
• Fix: Batch and File user level permissions by @Sameerlite in #19981
• [Feat]Add cost tracking and usage object in aretrieve_batch call type by @Sameerlite in #19986
• Add routing of xai chat completions to responses when web search options is present by @Sameerlite in #20051
• Add disable flag for anthropic gemini cache translation by @Sameerlite in #20052
• fix aspectRatio mapping in image edit by @Sameerlite in #20053
• Fix: vllm embedding format by @Sameerlite in #20056
• Fix: remove unsupported prompt-caching-scope-2026-01-05 header for vertex ai by @Sameerlite in #20058
• [Feature] UI - Usage: Model Breakdown Per Key by @yuneng-jiang in #20039
• Add mock client factory pattern and mock support for PostHog, Helicone, and Braintrust integrations by @AlexsanderHamir in #19707
• Realtime API benchmarks by @ishaan-jaff in #20074
• fixes: ci pipeline router coverage failure by @Harshit28j in #20065
• [cookbook] - add section for using claude agent sdk + MCPs with LiteLLM by @ishaan-jaff in #20081
• [Feat] Add async_post_call_response_headers_hook to CustomLogger by @ishaan-jaff in #20083
• fix(proxy): resolve high CPU when router_settings in DB by avoiding REGISTRY.collect() in PrometheusServicesLogger by @AlexsanderHamir in #20087
• Revert logs view commits by @ishaan-jaff in #20090
• [Fix] UI - Navbar: Fixed Default Logo + Bound Logo Box by @yuneng-jiang in #20092
• [Refactor] UI - Navbar: User Dropdown by @yuneng-jiang in #20095
• [Feat] v2 - Logs view with side panel and improved UX by @ishaan-jaff in #20091
• [Feat] UI - New View to render "Tools" on Logs View by @ishaan-jaff in #20093
• [Feat] UI - Add Pretty print view of request/response by @ishaan-jaff in #20096
• fixed mcp tools instructions on ui to show comma seprated str instead… by @shivamrawat1 in #20101
• litellm_fix: add missing timezone import to proxy_server.py by @shin-bot-litellm in #20121
• litellm_fix(proxy): reduce PLR0915 complexity (minimal) by @shin-bot-litellm in #20127
• litellm_fix(ui): remove unused ToolOutlined import by @shin-bot-litellm in #20129
• litellm_fix(e2e): disable bedrock-converse-claude-sonnet-4.5 in tests by @shin-bot-litellm in #20131
• litellm_fix(test): fix Azure AI cost calculator test - use Logging class by @shin-bot-litellm in #20134
• litellm_fix(test): fix Bedrock tool search header test regression by @shin-bot-litellm in #20135
• litellm_fix(test): allow comment field in schema and exclude robotics models from tpm check by @shin-bot-litellm in #20139
• litellm_docs: add missing environment variable documentation by @shin-bot-litellm in #20138
• litellm_fix(test): add acancel_batch to Azure SDK client initialization test by @shin-bot-litellm in #20143
• litellm_fix: handle unknown models in Azure AI cost calculator by @shin-bot-litellm in #20150
• litellm_fix(test): fix router silent experiment tests to properly mock async functions by @shin-bot-litellm in #20140
• [Feature] UI - Dark Mode: Delete Resource Modal by @yuneng-jiang in #20098
• [Fix] UI - Vector Store: Allow Config Defined Models to Be Selected by @yuneng-jiang in #20031
• [Fix] Add WATSONX_ZENAPIKEY to WatsonX credentials by @yuneng-jiang in #20086
• [Infa] UI Build by @yuneng-jiang in #20154
• litellm_fix: use get_async_httpx_client for logo download by @shin-bot-litellm in #20155
• litellm_fix: check for agent mode before requiring DD_API_KEY/DD_SITE by @shin-bot-litellm in #20156
• litellm_fix: handle empty dict for web_search_options in Nova grounding by @shin-bot-litellm in #20159
• litellm_mypy_fix_batch1: fix type errors in files, opentelemetry, gemini transformation, key management by @shin-bot-litellm in #20161
• litellm_fix(test): update Prometheus metric test assertions with new labels by @shin-bot-litellm in #20162
• litellm_fix: remove hosted_vllm from OpenAI client tests by @shin-bot-litellm in #20163
• litellm_fix: bump litellm-proxy-extras version to 0.4.28 by @shin-bot-litellm in #20166
• litellm_fix(mypy): fix remaining type errors by @shin-bot-litellm in #20164
• litellm_fix(security): allowlist Next.js CVEs for 7 days by @shin-bot-litellm in #20169
• litellm_fix(router): use safe_deep_copy in _get_silent_experiment_kwargs by @shin-bot-litellm in #20170
• docs(embeddings): add supported input formats section by @Chesars in #20073
• litellm_fix(lint): remove unused ToolNameValidationResult imports by @shin-bot-litellm in #20176
• litellm_fix(azure): Fix acancel_batch not using Azure SDK client initialization by @shin-bot-litellm in #20168
• [Fix] Model Name During Fallback by @yuneng-jiang in #20177
• [Fix] Health Endpoints when Callback Objects Defined by @yuneng-jiang in #20182
• fix(test): add router.acancel_batch coverage by @shin-bot-litellm in #20183
• fix(mypy): fix validate_tool_name return type signatures by @shin-bot-litellm in #20184
• litellm_fix_flaky_batch_completion_test by @shin-bot-litellm in #20186
• litellm_fix_cost_calc_test: correct prompt_tokens in test_string_cost_values by @shin-bot-litellm in #20185
• docs/blog index page by @ryan-crabbe in #20188
• [Docs] UI Spend Logs Settings Docs by @yuneng-jiang in #20197
• [Doc] Fixing Image by @yuneng-jiang in #20198
• litellm_fix_mapped_tests_core: clear client cache and fix isinstance checks by @shin-bot-litellm in #20196
• docs: fix dead links in v1.81.6 release notes by @shin-bot-litellm in #20218
• litellm_fix_mapped_tests_core: fix test isolation and mock injection issues by @shin-bot-litellm in #20209
• docs: Update v1.81.6 release notes - Logs v2 with Tool Call Tracing by @shin-bot-litellm in #20225
• feat: Support dimensions param for Cohere embed v4 by @amirzaushnizer in #20235
• feat: add User-Agent customization support by @jayy-77 in #19881
• feat(bedrock): add 1hr tiered caching costs for long-context models (#18988) by @cscguochang in #20214
• Update Vertex AI Text to Speech doc to show use of audio by @Sameerlite in #20255
• fix: add reasoning param support for GPT OSS cerebras by @Sameerlite in #20258
• Fix: Slack alert issue by @Sameerlite in #20257
• fix: Map reasoning content to anthropic thinking block(streaming+non-streaming) by @Sameerlite in #20254
• Fix open_ai_embedding_models to have custom_llm_provider None by @Sameerlite in #20253
• [Feat] Add support for 0 cost models by @Sameerlite in #20249
• Add Anthropic caching and context tests by @Sameerlite in #20247
• Litellm test bedrock optional params embeddings dimension by @Sameerlite in #20262
• [Feat]Add support for nova sonic Speech to speech model by @Sameerlite in #20244
• Litellm oss staging 01 31 2026 3 by @Sameerlite in #20266
• perf: optimize wrapper_async with CallTypes caching and reduced lookups by @ryan-crabbe in #20204
• perf: cache _get_relevant_args_to_use_for_logging() at module level by @ryan-crabbe in #20077
• docs: Add FAQ for setting up and verifying LITELLM_LICENSE by @shin-bot-litellm in #20284
• Model request tags documentation by @ishaan-jaff in #20290
• [Infra] UI - Update next to 16.1.6 by @yuneng-jiang in #20220
• [Fix] Remove Key Blocking on Login by @yuneng-jiang in #20210
• feat: add Kimi K2.5 model entries for Moonshot provider by @krauckbot in #20273
• fix: MCP "Session not found" error on VSCode reconnect by @shin-bot-litellm in #20298
• [Feature] SSO Config Team Mappings by @yuneng-jiang in #20111
• feat: add base /scim/v2 endpoint for SCIM resource discovery by @shin-bot-litellm in #20301
• docs: add Prisma migration troubleshooting guide by @shin-bot-litellm in #20300
• [Fix] UI - Team Settings: Disable Global Guardrail Persistence by @yuneng-jiang in #20307
• [Feature] Key reset_spend endpoint by @yuneng-jiang in #20305
• [Feature] UI - SSO: Add Team Mappings by @yuneng-jiang in #20299
• Add blog post: Achieving Sub-Millisecond Proxy Overhead by @AlexsanderHamir in #20309
• [Feat] - MCP Semantic Filtering Support by @ishaan-jaff in #20296
• Litellm docs mcp filtering semantic by @ishaan-jaff in #20316
• [Feat] /chat/completions - allow using OpenAI style tools for web_search with VertexAI/gemini models by @ishaan-jaff in #20280
• Add documentation correctly for nova sonic by @Sameerlite in #20320
• [Feature] UI - Default Team Settings: Migrate Default Team Settings to use Reusable Model Select by @yuneng-jiang in #20310
• [Feature] UI - Navbar: Option to Hide Community Engagement Buttons by @yuneng-jiang in #20308
• Litellm tuesday cicd release by @Sameerlite in #20328
• Revert "Litellm tuesday cicd release" by @Sameerlite in #20330
• Litellm tuesday cicd release final by @Sameerlite in #20333

New Contributors

• @zifeo made their first contribution in #19805
• @rushilchugh01 made their first contribution in #19938
• @cfchase made their first contribution in #19893
• @ayim made their first contribution in #19872
• @varunsripad123 made their first contribution in #20018
• @nht1206 made their first contribution in #20046
• @genga6 made their first contribution in #20009
• @shin-bot-litellm made their first contribution in #20121
• @amirzaushnizer made their first contribution in #20235
• @cscguochang made their first contribution in #20214

Full Changelog: v1.81.3.rc.5...v1.81.7-nightly