💵 LiteLLM v1.38.11 Proxy 100+ LLMs AND Set Budgets for your customers https://docs.litellm.ai/docs/proxy/users#set-rate-limits
✨ NEW /Customer/update and /Customer/delete endpoints https://docs.litellm.ai/docs/proxy/users#set-rate-limits
📝 [Feat] Email alerting is now Free Tier: https://docs.litellm.ai/docs/proxy/email
🚀 [Feat] Show supported OpenAI params on LiteLLM UI model hub
✨ [Feat] Show Created at, Created by on Models Page
What's Changed
- Clarifai-LiteLLM update docs by @mogith-pn in #3856
- [Feat] Show supported OpenAI params on model hub by @ishaan-jaff in #3859
- fix(parallel_request_limiter.py): fix user+team tpm/rpm limit check by @krrishdholakia in #3857
- fix - Admin UI show activity by model_group by @ishaan-jaff in #3865
- [Feat] Show Created at, Created by on
Models
Page by @ishaan-jaff in #3868 - Improve validate-fallbacks method by @SujanShilakar in #3847
- fix(model_dashboard.tsx): accurately show the input/output cost per token when custom pricing is set by @krrishdholakia in #3871
- Admin UI - Public model hub by @krrishdholakia in #3869
- [Feat] Rename
/end/user/new
->/customer/new
(maintain backwards compatibility) by @ishaan-jaff in #3870 - [Feat] Make Email alerting Free Tier, but customizing emails enterprise by @ishaan-jaff in #3872
New Contributors
- @SujanShilakar made their first contribution in #3847
Full Changelog: v1.38.10...v1.38.11
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.38.11
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 94 | 113.13091035154665 | 6.485092627447978 | 0.0 | 1940 | 0 | 80.4994959999874 | 735.4111310000064 |
Aggregated | Passed ✅ | 94 | 113.13091035154665 | 6.485092627447978 | 0.0 | 1940 | 0 | 80.4994959999874 | 735.4111310000064 |