BerriAI/litellm v1.72.6.rc on GitHub

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.72.6.rc

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	190.0	206.1983524503451	6.292694671664114	0.0	1883	0	168.78388700001778	1487.5943659999962
Aggregated	Passed ✅	190.0	206.1983524503451	6.292694671664114	0.0	1883	0	168.78388700001778	1487.5943659999962

What's Changed

[Docs] v1.72.2.rc by @ishaan-jaff in #11519
Support env var vertex credentials for passthrough + ignore space id on watsonx deployment (throws Json validation errors) by @krrishdholakia in #11527
Ensure consistent 'created' across all chunks + set tool call id for ollama streaming calls by @krrishdholakia in #11528
Update enduser spend and budget reset date based on budget duration by @laurien16 in #8460
feat: add .cursor to .gitignore by @colesmcintosh in #11538
Add gpt-4o-audio-preview-2025-06-03 pricing configuration by @colesmcintosh in #11560
[Docs] Fix incorrect reference to database_url as master_key by @fengbohello in #11547
Update documentation for configuring web search options in config.yaml by @colesmcintosh in #11537
[Bug fix]: aiohttp fixes for transfer encoding error on aiohttp transport by @ishaan-jaff in #11561
[Feat] Add reasoning_effort support for perplexity models by @ishaan-jaff in #11562
Make all commands show server URL by @msabramo in #10801
Simplify management_cli.md CLI docs by @msabramo in #10799
Fix: Adds support for choosing the default region based on where the model is available by @ishaan-jaff in #11566
[Feat] Add Lasso Guardrail to LiteLLM by @ishaan-jaff in #11565
Fix gemini tool call indexes by @lowjiansheng in #11558
Show remaining users on UI + prevent early stream stopping for gemini requests by @krrishdholakia in #11568
Add VertexAI claude-opus-4 + Assign users to orgs on creation by @krrishdholakia in #11572
Pangea/kl/udpate readme by @lapinek in #11570
Update README.md so docker compose will work as described by @yanwork in #11586
Add support for new Mistral Magistral models (magistral-medium-2506 and magistral-small-2506) by @colesmcintosh in #11588
(fix:exception_mapping_utils.py) fix sglang rate limit error issue by @dhs-shine in #11575
[Feat] LiteLLM Allow setting Uvicorn Keep Alive Timeout by @ishaan-jaff in #11594
[Bug Fix] No module named 'diskcache' by @ishaan-jaff in #11600
[Feat] UI - Add controls for MCP Permission Management by @ishaan-jaff in #11598
[Feat] New LLM API Endpoint - Add List input items for Responses API by @ishaan-jaff in #11602
Add new o3 models pricing by @krrishdholakia in #11606
[UI] Polish New MCP Server Add Form by @ishaan-jaff in #11604
Litellm dev 06 10 2025 p2 by @krrishdholakia in #11605
Add VertexAI Anthropic passthrough - cost calculation, token tracking by @krrishdholakia in #11611
fix(internal_user_endpoints.py): support user with + in email on user info + handle empty string for arguments on gemini function calls by @krrishdholakia in #11601
Fix: passes api_base, api_key, litellm_params_dict to custom_llm embedding methods by @ElefHead in #11450
Add Admin-Initiated Password Reset Flow by @NANDINI-star in #11618
fix inference endpoints (#11630) by @ishaan-jaff in #11631
[UI] Add Deepgram provider to supported providers list and mappings by @ishaan-jaff in #11634
[Bug Fix] Add audio/ogg mapping for Audio MIME types by @ishaan-jaff in #11635
[Feat] Add Background mode for Responses API - OpenAI, AzureOpenAI by @ishaan-jaff in #11640
[Feat] Add provider specific params for deepgram/ by @ishaan-jaff in #11638
[Feat] MCP - Add support for streamablehttp_client MCP Servers by @ishaan-jaff in #11628
[Feat] Perf fix - ensure deepgram provider uses async httpx calls by @ishaan-jaff in #11641
Trim the long user ids on the the keys page by @NANDINI-star in #11488
Enable System Proxy Support for aiohttp Transport by @idootop in #11616
GA Multi-instance rate limiting v2 Requirements + New - specify token rate limit type - output / input / total by @krrishdholakia in #11646
Add bridge for /chat/completion -> /responses API by @krrishdholakia in #11632
Convert scientific notation str to int + Bubble up azure content filter results by @krrishdholakia in #11655
feat(helm): [#11648] support extraContainers in migrations-job.yaml by @stevenaldinger in #11649
Correct success message when user creates new budget by @vuanhtu52 in #11608
fix: Do not add default model on tag based-routing when valid tag by @thiagosalvatore in #11454
Fix default user settings by @NANDINI-star in #11674
[Pricing] add azure/gpt-4o-mini-transcribe models by @ishaan-jaff in #11676
Enhance Mistral model support with reasoning capabilities by @colesmcintosh in #11642
[Feat] MCP expose streamable https endpoint for LiteLLM Proxy by @ishaan-jaff in #11645
change space_key header to space_id for Arize by @vanities in #11595
Add performance indexes to LiteLLM_SpendLogs for analytics queries by @colesmcintosh in #11675
Revert "Add performance indexes to LiteLLM_SpendLogs for analytics queries" by @krrishdholakia in #11683
[Feat] Use dedicated Rest endpoints for list, calling MCP tools by @ishaan-jaff in #11684
Chat Completions <-> Responses API Bridge Improvements by @krrishdholakia in #11685
[UI] Fix MCP Server Table to Match Existing Table Pattern by @ishaan-jaff in #11691
Logging: prevent double logging logs when bridge is used (anthropic <-> chat completion OR chat completion <-> responses api) by @krrishdholakia in #11687
fix(vertex_ai): support global location in vertex ai passthrough by @alvarosevilla95 in #11661
[Feat] UI Allow editing mcp servers by @ishaan-jaff in #11693
[Feat] UI - Allow setting MCP servers when creating keys, teams by @ishaan-jaff in #11711
[Feat] Add Authentication + Permission Management for MCP List, Call Tool Ops by @ishaan-jaff in #11682
Add Live Tail Feature to Logs View by @NANDINI-star in #11712
[Feat] Add Connect to MCP Page by @ishaan-jaff in #11716
Enterprise feature preview improvement on Audit Logs by @NANDINI-star in #11715
Align Model Connection Success Icon and Text by @NANDINI-star in #11717
fix(prometheus.py): fix total requests increment + add semantic tests fo… by @krrishdholakia in #11718
Anthropic - add 'prefix' to start of assistant content + Add model access groups on UI by @krrishdholakia in #11719
Add anthropic 'none' tool choice param support by @krrishdholakia in #11695
[Feat] UI - Add back favicon by @ishaan-jaff in #11728
Time taken column logs by @gbrian in #11723
UI - Fix remaining users activity if no limit + allow filtering by model access groups by @krrishdholakia in #11730
[Feat] Prometheus - Allow controlling exposed metrics and labels by @ishaan-jaff in #11489
Add Langfuse OpenTelemetry Integration by @colesmcintosh in #11607
Fixed grok-3-mini to not use stop tokens by @Naggafin in #11563
Implement health check backend API and storage functionality by @colesmcintosh in #11678
fix: re-export get_valid_models utils in init file by @jsondai in #11721
[Bug Fix] Ensure /redis/info works on GCP Redis by @ishaan-jaff in #11732
[Fix] JWT - Fix error when team member already part of team by @ishaan-jaff in #11735
[Bug Fix] Rollback dd-trace version by @ishaan-jaff in #11736
Prometheus - fix request increment + add route tracking for streaming requests by @krrishdholakia in #11731
Litellm stable release 06 14 2025 by @krrishdholakia in #11737

New Contributors

@laurien16 made their first contribution in #8460
@fengbohello made their first contribution in #11547
@lapinek made their first contribution in #11570
@yanwork made their first contribution in #11586
@dhs-shine made their first contribution in #11575
@ElefHead made their first contribution in #11450
@idootop made their first contribution in #11616
@stevenaldinger made their first contribution in #11649
@thiagosalvatore made their first contribution in #11454
@vanities made their first contribution in #11595
@alvarosevilla95 made their first contribution in #11661
@gbrian made their first contribution in #11723
@Naggafin made their first contribution in #11563
@jsondai made their first contribution in #11721

Full Changelog: v1.72.2-stable...v1.72.6.rc