What's Changed
- [UI-Polish] Cleanup Inputing Key Name, Team Name, User Email by @ishaan-jaff in #3218
- Update langsmith_integration.md by @bllchmbrs in #3205
- Added openrouter/meta-llama/llama-3-70b-instruct context and cost metrics by @paul-gauthier in #3223
- [UI-Fix] Show all teams on Admin UI by @ishaan-jaff in #3219
- fix(prometheus.py): add user tracking to prometheus by @krrishdholakia in #3224
- [Bug-Fix] Alerting - don't send hanging request alert on failed request by @ishaan-jaff in #3226
- improve(vertex_ai.py): Switch to simpler dict type for supporting JSON mode by @Manouchehri in #3211
- (Vertex AI) - Add
frequency_penaltyandpresence_penaltysupport by @Manouchehri in #3214 - [Fix] Non-Admin SSO Login by @ishaan-jaff in #3228
New Contributors
- @bllchmbrs made their first contribution in #3205
- @paul-gauthier made their first contribution in #3223
Full Changelog: 1.35.19.dev1...v1.35.20
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 56 | 66.18343163694331 | 1.5732072585426606 | 0.003340142799453632 | 471 | 1 | 50.73058199997149 | 1221.7388810000216 |
| /health/liveliness | Passed ✅ | 40 | 43.59827081867388 | 15.418099162277965 | 0.0 | 4616 | 0 | 37.99386999997978 | 1076.7643240000098 |
| /health/readiness | Passed ✅ | 40 | 43.16513574967192 | 15.264452593503098 | 0.0 | 4570 | 0 | 38.20107499996084 | 1247.142073999953 |
| Aggregated | Passed ✅ | 40 | 44.49484154250821 | 32.25575901432372 | 0.003340142799453632 | 9657 | 1 | 37.99386999997978 | 1247.142073999953 |