What's Changed
- (feat) litellm proxy /cache/ping by @ishaan-jaff in #2602
Full Changelog: v1.32.7...v1.32.7.dev1
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 61 | 63.516820718279114 | 1.5532993244098399 | 1.5532993244098399 | 465 | 465 | 59.33860800001867 | 610.6795680000232 |
| /health/liveliness | Failed ❌ | 61 | 62.914033896878706 | 15.516291100825175 | 15.516291100825175 | 4645 | 4645 | 58.9269360000344 | 1474.1329439999618 |
| /health/readiness | Failed ❌ | 61 | 63.107874450011415 | 15.269099380381459 | 15.269099380381459 | 4571 | 4571 | 58.92951800001356 | 1356.0212780000143 |
| Aggregated | Failed ❌ | 61 | 63.03451122776627 | 32.338689805616475 | 32.338689805616475 | 9681 | 9681 | 58.9269360000344 | 1474.1329439999618 |