Full Changelog: v1.35.23...v1.35.24.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 96 | 104.08889620803765 | 1.4128712196268784 | 0.0 | 423 | 0 | 90.27053599999135 | 1337.9505959999847 |
| /health/liveliness | Passed ✅ | 79 | 81.64779973608695 | 15.605045716540841 | 0.0 | 4672 | 0 | 77.07790199998499 | 1228.4024810000176 |
| /health/readiness | Passed ✅ | 79 | 83.50619211328004 | 14.890259804010931 | 0.00668024217317673 | 4458 | 2 | 77.08443000001353 | 1429.602591000048 |
| Aggregated | Passed ✅ | 79 | 83.50871222694458 | 31.90817674017865 | 0.00668024217317673 | 9553 | 2 | 77.07790199998499 | 1429.602591000048 |