What's Changed
- [Fix] pip litellm proxy - No module named 'nacl' by @ishaan-jaff in #5353
- build(deps): bump micromatch from 4.0.5 to 4.0.8 in /ui by @dependabot in #5349
- Admin UI - show user_id on Key Table by @ishaan-jaff in #5350
- [Feat-Vertex] Support using workload identity federation by @ishaan-jaff in #5354
- [Feat-Caching] allow setting caching mode to default off by @ishaan-jaff in #5352
- Feat - ui allow setting tpm / rpm limits on keys on ui by @ishaan-jaff in #5356
- [Feat] Admin UI - allow setting tpm / rpm per model by @ishaan-jaff in #5357
- refactor(main.py): migrate vertex gemini calls to vertex_httpx by @krrishdholakia in #4582
Full Changelog: v1.44.5...1.44.6
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-1.44.6
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 86 | 114.28056593503096 | 6.432087013394555 | 0.0033430805682923884 | 1924 | 1 | 67.2053769999934 | 8988.791535000018 |
| Aggregated | Passed ✅ | 86 | 114.28056593503096 | 6.432087013394555 | 0.0033430805682923884 | 1924 | 1 | 67.2053769999934 | 8988.791535000018 |