What's Changed
- [UI] show exceptions by model deployments + model latencies - v0 by @ishaan-jaff in #3373
- [UI] Polish viewing Model Latencies by @ishaan-jaff in #3380
Full Changelog: v1.35.33...v1.35.33.dev2
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 81 | 88.56872818162829 | 1.6000195569734517 | 0.0 | 479 | 0 | 75.49507499999208 | 1089.8572720000175 |
| /health/liveliness | Passed ✅ | 65 | 68.32578986950885 | 15.435679275102965 | 0.0033403331043287093 | 4621 | 1 | 63.41570199998614 | 1720.5565799999931 |
| /health/readiness | Passed ✅ | 66 | 69.06075345009796 | 15.29538528472116 | 0.010020999312986128 | 4579 | 3 | 63.36736399998699 | 1532.7052860000094 |
| Aggregated | Passed ✅ | 66 | 69.67528523959074 | 32.33108411679758 | 0.013361332417314837 | 9679 | 4 | 63.36736399998699 | 1720.5565799999931 |