BerriAI/litellm v1.80.7.dev.3 on GitHub

What's Changed

[Infra] Building UI by @yuneng-jiang in #17212
[Infra] Rebuilding UI by @yuneng-jiang in #17213
docs: add OpenAI Agents SDK to projects by @Chesars in #17203
Upgrade websockets to v15 by @hxyannay in #16734
fix: do not include plaintext message in exception by @raghav-stripe in #17216
[Fix] Change Add Fallback Modal to use Antd Select by @yuneng-jiang in #17223
[Feat] Add guardrails for pass through endpoints by @ishaan-jaff in #17221
[Feat] UI - allow adding pass through guardrails through UI by @ishaan-jaff in #17226
[Bug fix] Vector Store List Endpoint Returns 404 by @ishaan-jaff in #17229
[Feature]: Add Provider publicai.co by @ishaan-jaff in #17230
[Fix] Request and Response Panel JSONViewer by @yuneng-jiang in #17233
[Feature] Adding Button Loading States to Edit Settings by @yuneng-jiang in #17236
[Fix] Various Text, button state, and test changes by @yuneng-jiang in #17237
[Fix] UI - Fallbacks Immediately Deleting before API resolves by @yuneng-jiang in #17238
[Fix] Remove Feature Flags by @yuneng-jiang in #17240
(Bug)Migrate Anthropic provider to azure ai by @Sameerlite in #17202
Fix/issue 16759 streaming error validation by @weichiet in #17242
feat: add regex-based tool_name/tool_type matching for tool-permission by @uc4w6c in #17164
SSO: fix the generic SSO provider by @saar-win in #17227
fix(bedrock): handle cohere v4 embed response dictionary format by @AndyForest in #17220
Fix : acompletion throws error with SambaNova models by @omkar806 in #17217
Allow wildcard routes for nonproxy admin (SCIM) by @v0rtex20k in #17178
Update new anthropic feats as reviewed by @Sameerlite in #17142
Fix metadata tags and model name display in UI for Azure passthrough + Add cost tracking for responses API by @Sameerlite in #17258
Add nova embedding support by @Sameerlite in #17253
Better handle anonymization by @hxomer in #17207
Added support for twelvelabs pegasus by @Sameerlite in #17193
[Refactor] Deprecate spend/logs & add spend/logs/v2 by @AlexsanderHamir in #17167
[Perf] Fix /audio/speech performance by using shared_sessions by @AlexsanderHamir in #16739
Respect custom llm provider in header by @Sameerlite in #17290
(Feat)Add passthrough cost tracking for veo by @Sameerlite in #17296
Remove not compatible beta header from Bedrock by @Sameerlite in #17301
Add better handling in image generation for gemini models by @Sameerlite in #17292
fix: respect guardrail mock_response during during_call to return blo… by @uc4w6c in #17247
Fix session consistency, move Lasso API version away from source code by @orgersh92 in #17316
doc: add images for tool permission guardrail by @uc4w6c in #17322
[Feat] WatsonX - allow passing zen_api_key dynamically by @ishaan-jaff in #16655
[Feat] JWT Auth - AI Gateway, allow using regular OIDC flow with user info endpoints by @ishaan-jaff in #17324
[Feat] Add new modelfireworks_ai /kimi-k2-instruct-0905 by @ishaan-jaff in #17328
(feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo by @krrishdholakia in #17175
Revert "fix: respect guardrail mock_response during during_call to return blo…" by @krrishdholakia in #17332
Add claude-opus-4-5 alias to pricing data by @dannykopping in #17313
Add support of audio transcription for OVHcloud by @eliasto in #17305
[Fix] Fix Watsonx Audio Transcription API by @ishaan-jaff in #17326
[fix] extra_headers in messages api bedrock invoke by @colinlin-stripe in #17271
fix: GA path for azure openai realtime models by @abi-jey in #17260
[Bug Fix] - Fix litellm_enterprise ensure imported routes exist by @ishaan-jaff in #17337
Fix/new org team validate against org by @rioiart in #17333
Fix sso users not added to entra synced team by @rioiart in #17331
Refactor Noma guardrail to use shared Responses transformation and include system instructions by @idola9 in #17315
feat(provider): add Z.AI (Zhipu AI) as built-in provider by @Chesars in #17307
Fix AttributeError when metadata is null in request body by @Chesars in #17306
SSO(fix): Clear SSO integration for all users by @saar-win in #17287
feat: add experimental latest-user filtering for Bedrock by @uc4w6c in #17282
Add context window exception mapping for Together AI by @li-boxuan in #17284
feat(github-copilot): Add Embedding API support by @codgician in #17278
Update Databricks model pricing and add new models (including databricks pricing test). by @epistoteles in #17277
fix: remove URL format validation for MCP server endpoints by @uc4w6c in #17270
Guardrail API V2 - user api key metadata, session id, specify input type (request/response), image support by @krrishdholakia in #17338
fix: update default database connection number by @AlexsanderHamir in #17353
Fix: update default proxy_batch_write_at number by @AlexsanderHamir in #17355
[Feat] Add vllm batch+files API support by @ishaan-jaff in #15823
Fix 500 error for malformed request by @Sameerlite in #17291
Fix: litellm user auth not passing issue by @Sameerlite in #17342
Add other routes in jwt auth by @Sameerlite in #17345
docs: add Azure AI Foundry documentation for Claude models by @Chesars in #17104
Litellm bedrock OpenAI model support by @kothamah in #17368
[Fix] Bedrock Guardrail Indent and Import fix by @yuneng-jiang in #17378
[Infra] Add Google Private API Endpoint to Vertex AI fields by @yuneng-jiang in #17382
[Docs] Add guide on how to debug gateway error vs provider error by @ishaan-jaff in #17387
🐛 Fix proxy caching between requests in aiohttp transport by @lcfyi in #17122
[Feat] New model/provider - Adds support for Google Cloud Chirp3 HD on /speech by @ishaan-jaff in #17391
[Fix] UI - Change Edit Team Models Shown to Match Create Team by @yuneng-jiang in #17394
[New model] Add fireworks/deepseek-v3p2 by @ishaan-jaff in #17395
feat: Add Nova lite 2 reasoning support with reasoningConfig by @flozonn in #17371
[Fix] Show all credential values on Edit Credential Modal by @yuneng-jiang in #17397
[Refactor] Standardize API Key vs Virtual Key in UI by @yuneng-jiang in #17325
[Feature] Add User Alias Column to Internal User Table by @yuneng-jiang in #17321
Fix: Datadog callback regression when ddtrace is installed by @ishaan-jaff in #17393
Refactor add_schema_to_components to move definitions to components/s… by @richardmcsong in #17389
Support Deepseek 3.2 with Reasoning by @matt-greathouse in #17384
Add model price & details for Global Opus 4.5 Bedrock endpoint by @mossbanay in #17380
docs: Update Instructions For Phoenix Integration by @ialisaleh in #17373
update model_prices_and_context_window.json by @mhielpos-asapp in #17376
fix: conditionally pass enable_cleanup_closed to aiohttp TCPConnector by @Joilence in #17367
fix: replace deprecated .dict() with .model_dump() in streaming_handler by @Joilence in #17359
feat: make streaming chunk size configurable in bedrock api by @deepaktammali in #17357
Cleanup: Remove orphan docs pages and Docusaurus template files by @Chesars in #17356
docs: add Google ADK and Harbor to projects by @Chesars in #17352
Helm Chart: add ingress-only labels by @freinold in #17348
Fix/organization max budget not enforced by @rioiart in #17334
Litellm chainguard fixes 12 02 2025 p1 by @krrishdholakia in #17406
Guardrails API - add streaming support by @krrishdholakia in #17400
Add ragflow support for chat completions API by @Sameerlite in #17350
Make thought sign in tool call id as a beta feat by @Sameerlite in #17405
Enforce support of enforce_user_param to openai post endpoints by @Sameerlite in #17407
Fix bedrock models in model map by @Sameerlite in #17419
Fix gemini 3 last chunk thinking block by @Sameerlite in #17403
Add vector store support for ragflow by @Sameerlite in #17414
docs: add Microsoft Agent Lightning to projects by @Chesars in #17422
fix: prevent memory leak in aiohttp connection pooling by @AlexsanderHamir in #17388
adding status parameter as optinal for FileObject by @FelipeRodriguesGare in #17431
Batches API - fixing optional parameter default value by @FelipeRodriguesGare in #17434
[Refactor] /login route by @yuneng-jiang in #17379
[Refactor#2] litellm/init – Lazy-load utils to reduce memory + import time by @AlexsanderHamir in #17171
[Fix] Remove second scrollbar when sidebar is expanded + tooltip z index by @yuneng-jiang in #17436
[Feature] Delete Credential Enhancements by @yuneng-jiang in #17317
[Bug Fix] Parallel Request Limiter with /messages by @ishaan-jaff in #17426
Bump mcp from 1.10.1 to 1.23.0 in /.circleci by @dependabot[bot] in #17363
[Fix] Change e2e test to look for Virtual Keys instead of API Keys by @yuneng-jiang in #17446
[Feature] Add auto_redirect_to_sso to UI Config by @yuneng-jiang in #17399
[Feature] New Login Page by @yuneng-jiang in #17443
[Fix] Change labeling around Vertex Fields by @yuneng-jiang in #17383
[Fix] Return 404 when a user is not found on /user/info by @yuneng-jiang in #16850
fix: cyberark allow setting ssl verfiy to false by @ishaan-jaff in #17433
[Feat] Agent Gateway - Allow invoking agents through AI Gateway by @ishaan-jaff in #17440
[Feat] Agent Gateway - Allow tracking request / response in "Logs" Page by @ishaan-jaff in #17449
[Docs] Agent Gateway by @ishaan-jaff in #17454
[Feat] UI - Agent Gateway - allow testing agents on UI by @ishaan-jaff in #17455
Fix: Allow reasoning_effort='none' for Azure gpt-5.1 models by @Chesars in #17311
Revert "Bump mcp from 1.10.1 to 1.23.0 in /.circleci" by @ishaan-jaff in #17457
fix: clean arize-phoenix traces by @axiomofjoy in #16611
fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail by @krrishdholakia in #17424

New Contributors

@hxyannay made their first contribution in #16734
@weichiet made their first contribution in #17242
@AndyForest made their first contribution in #17220
@omkar806 made their first contribution in #17217
@v0rtex20k made their first contribution in #17178
@hxomer made their first contribution in #17207
@orgersh92 made their first contribution in #17316
@dannykopping made their first contribution in #17313
@rioiart made their first contribution in #17333
@epistoteles made their first contribution in #17277
@kothamah made their first contribution in #17368
@flozonn made their first contribution in #17371
@richardmcsong made their first contribution in #17389
@matt-greathouse made their first contribution in #17384
@mossbanay made their first contribution in #17380
@mhielpos-asapp made their first contribution in #17376
@Joilence made their first contribution in #17367
@deepaktammali made their first contribution in #17357
@axiomofjoy made their first contribution in #16611

Full Changelog: v1.80.6-nightly...v1.80.7.dev.3