What's Changed
- [Fix] Lowest Latency routing - random pick deployments when all latencies=0 by @ishaan-jaff in #3360
Full Changelog: v1.35.32...v1.35.32.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 56 | 60.03167063674528 | 1.6000338631233044 | 0.0 | 479 | 0 | 49.292497999999796 | 574.5523299999888 |
| /health/liveliness | Passed ✅ | 40 | 42.727974751455534 | 15.495943822607535 | 0.0 | 4639 | 0 | 37.59355699997968 | 1116.836471000056 |
| /health/readiness | Passed ✅ | 40 | 42.65787459895715 | 15.37569075565046 | 0.0 | 4603 | 0 | 37.80075800000304 | 1220.3729720000354 |
| Aggregated | Passed ✅ | 41 | 43.54741712642739 | 32.4716684413813 | 0.0 | 9721 | 0 | 37.59355699997968 | 1220.3729720000354 |