What's Changed
- (feat) litellm proxy /cache/ping by @ishaan-jaff in #2602
Full Changelog: v1.32.7...v1.32.7.dev5
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 85 | 96.88052374178396 | 1.4227971757149624 | 0.0 | 426 | 0 | 79.01798800003235 | 1664.4016860000193 |
| /health/liveliness | Passed ✅ | 62 | 65.33972411722846 | 15.470414361295084 | 0.0 | 4632 | 0 | 60.18862700000227 | 1455.14007700001 |
| /health/readiness | Passed ✅ | 62 | 66.49270392100088 | 15.473754260768594 | 0.0 | 4633 | 0 | 60.21157800000765 | 1677.9052240000283 |
| Aggregated | Passed ✅ | 62 | 67.27741228882489 | 32.36696579777864 | 0.0 | 9691 | 0 | 60.18862700000227 | 1677.9052240000283 |