What's Changed
- Add support for new Mistral Magistral models (magistral-medium-2506 and magistral-small-2506) by @colesmcintosh in #11588
- (fix:exception_mapping_utils.py) fix sglang rate limit error issue by @dhs-shine in #11575
- [Feat] LiteLLM Allow setting Uvicorn Keep Alive Timeout by @ishaan-jaff in #11594
- [Bug Fix] No module named 'diskcache' by @ishaan-jaff in #11600
- [Feat] UI - Add controls for MCP Permission Management by @ishaan-jaff in #11598
- [Feat] New LLM API Endpoint - Add List input items for Responses API by @ishaan-jaff in #11602
- Add new o3 models pricing by @krrishdholakia in #11606
- [UI] Polish New MCP Server Add Form by @ishaan-jaff in #11604
- Litellm dev 06 10 2025 p2 by @krrishdholakia in #11605
- Add VertexAI Anthropic passthrough - cost calculation, token tracking by @krrishdholakia in #11611
New Contributors
- @dhs-shine made their first contribution in #11575
Full Changelog: v1.72.3-nightly...v1.72.4-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.72.4-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 180.0 | 204.8932724751626 | 6.202890810178717 | 0.0 | 1852 | 0 | 168.13937000000578 | 1311.1876840000036 |
| Aggregated | Passed ✅ | 180.0 | 204.8932724751626 | 6.202890810178717 | 0.0 | 1852 | 0 | 168.13937000000578 | 1311.1876840000036 |