Launching support for using LiteLLM LLM Gateway with Oauth2 proxy authentication
🛠️ Security Fix - don't allow clients to set api_base, base_url for requests to LiteLLM Proxy
🔥 Use ssml input for Vertex Text to Speech APIs: https://docs.litellm.ai/docs/providers/vertex#usage---ssml-as-input
⚡️ [UI] Allow viewing / editing budget duration
🛠️ [minor fix Proxy] - prometheus fix use safe update start / end time
What's Changed
- Support for gemini experimental models by @lilleswing in #5413
- [UI] Allow editing budget duration by @ishaan-jaff in #5409
- Feat - Add Google Text-to-Speech support ssml by @ishaan-jaff in #5415
- Make helm chart listen on IPv6 (and IPv4). by @ArbitraryCritter in #5222
- [Fix - Proxy] Security fix, don't allow client side to set api_base, base_url by @ishaan-jaff in #5418
- [Feat-Proxy] Add hook for oauth2 proxy headers by @ishaan-jaff in #5420
- [minor fix Proxy] - prometheus - safe update start / end time by @ishaan-jaff in #5422
New Contributors
- @lilleswing made their first contribution in #5413
- @ArbitraryCritter made their first contribution in #5222
Full Changelog: v1.44.8...v1.44.9
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.9
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 140.0 | 159.56966948018703 | 6.410714219145582 | 0.0 | 1918 | 0 | 110.58400799998935 | 1685.0977490000218 |
| Aggregated | Passed ✅ | 140.0 | 159.56966948018703 | 6.410714219145582 | 0.0 | 1918 | 0 | 110.58400799998935 | 1685.0977490000218 |