What's Changed
- Support for gemini experimental models by @lilleswing in #5413
- [UI] Allow editing budget duration by @ishaan-jaff in #5409
- Feat - Add Google Text-to-Speech support ssml by @ishaan-jaff in #5415
- Make helm chart listen on IPv6 (and IPv4). by @ArbitraryCritter in #5222
- [Fix - Proxy] Security fix, don't allow client side to set api_base, base_url by @ishaan-jaff in #5418
- [Feat-Proxy] Add hook for oauth2 proxy headers by @ishaan-jaff in #5420
- [minor fix Proxy] - prometheus - safe update start / end time by @ishaan-jaff in #5422
New Contributors
- @lilleswing made their first contribution in #5413
- @ArbitraryCritter made their first contribution in #5222
Full Changelog: v1.44.8...v1.44.8-dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.8-dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.8-dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 140.0 | 165.5998142681111 | 6.323551839446042 | 0.0 | 1891 | 0 | 107.57471499999838 | 2245.6900200000405 |
| Aggregated | Passed ✅ | 140.0 | 165.5998142681111 | 6.323551839446042 | 0.0 | 1891 | 0 | 107.57471499999838 | 2245.6900200000405 |