What's Changed
- UI select start/ end time for viewing model metrics by @ishaan-jaff in #3441
- Improve the document of Traceloop by @paneru-rajan in #3445
- Improve mocking in
test_proxy_exception_mapping.pyby @msabramo in #3408
New Contributors
- @paneru-rajan made their first contribution in #3445
Full Changelog: v1.35.38...v1.35.38-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 55 | 58.640919096329625 | 1.4563005047336437 | 0.0 | 436 | 0 | 50.270481999973526 | 588.9671030000159 |
| /health/liveliness | Passed ✅ | 40 | 43.084301563953716 | 15.5116044586767 | 0.0 | 4644 | 0 | 37.74072900000647 | 1439.319705999992 |
| /health/readiness | Passed ✅ | 40 | 43.8615925310188 | 15.398039740417655 | 0.006680277544649742 | 4610 | 2 | 37.88508399998136 | 1424.1174170000193 |
| Aggregated | Passed ✅ | 40 | 44.15406385521118 | 32.365944703828 | 0.006680277544649742 | 9690 | 2 | 37.74072900000647 | 1439.319705999992 |