What's Changed
- [Feat] Proxy - /cache/flushall - delete all elements from cache by @ishaan-jaff in #2702
- (fix) Proxy fix cache control logic - fix
no-storelogic by @ishaan-jaff in #2696 - [Fix] remove litellm telemetry from proxy server by @ishaan-jaff in #2703
Full Changelog: v1.34.4.dev1...v1.34.4.dev2
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 92 | 99.04622922560984 | 1.6434154232887563 | 0.0 | 492 | 0 | 86.46199399998977 | 496.77466399998593 |
| /health/liveliness | Passed ✅ | 78 | 85.9373431366058 | 15.037919178142237 | 0.0 | 4502 | 0 | 73.93678500000078 | 1486.8835759999683 |
| /health/readiness | Passed ✅ | 78 | 84.57857246170724 | 15.265057895182146 | 0.003340275250586903 | 4570 | 1 | 73.80300699998088 | 1357.8377890000013 |
| Aggregated | Passed ✅ | 79 | 85.96243619092432 | 31.94639249661314 | 0.003340275250586903 | 9564 | 1 | 73.80300699998088 | 1486.8835759999683 |