What's Changed
- fix(router.py): initial commit for semaphores on router by @krrishdholakia in #2996
Full Changelog: v1.35.3...v1.35.4
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 55 | 61.81466973473701 | 1.5865369739586979 | 0.0 | 475 | 0 | 50.56273999997529 | 1291.203577999994 |
| /health/liveliness | Passed ✅ | 40 | 43.40270568389331 | 15.511321488556195 | 0.0 | 4644 | 0 | 38.63372099999651 | 1381.9246239999927 |
| /health/readiness | Passed ✅ | 41 | 42.992702731896564 | 15.497961177196542 | 0.003340077839913048 | 4640 | 1 | 38.45442200000093 | 1314.9427230000015 |
| Aggregated | Passed ✅ | 41 | 44.103932164771024 | 32.595819639711436 | 0.003340077839913048 | 9759 | 1 | 38.45442200000093 | 1381.9246239999927 |