What's Changed
- [UI] Polish viewing Model Latencies by @ishaan-jaff in #3380
Full Changelog: v1.35.33.dev3...1.35.33.dev4
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 78 | 86.64201385977066 | 1.4529460265759184 | 0.0 | 435 | 0 | 71.46099299995967 | 1225.4950830000553 |
| /health/liveliness | Passed ✅ | 62 | 65.89392386120038 | 15.64171549989661 | 0.003340105808220502 | 4683 | 1 | 59.578546000011556 | 1532.8349000000117 |
| /health/readiness | Passed ✅ | 62 | 65.34362750907113 | 15.464689892060925 | 0.0 | 4630 | 0 | 59.670154999992064 | 1263.970363999988 |
| Aggregated | Passed ✅ | 62 | 66.5584239677881 | 32.55935141853345 | 0.003340105808220502 | 9748 | 1 | 59.578546000011556 | 1532.8349000000117 |