🚨 Stable Release: v1.34.1
What's Changed
- Litellm jwt auth fixes by @krrishdholakia in #2687
- [FEAT] batch write redis cache by @ishaan-jaff in #2693
- [Fix] better cache debugging/error statements by @ishaan-jaff in #2694
- (test) Using Cache controls with litellm -
no-cache=Trueby @ishaan-jaff in #2695
Full Changelog: v1.34.1...v1.34.3
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 92 | 95.53184210222173 | 1.5031119149701597 | 0.0 | 450 | 0 | 86.54108700000052 | 532.0916549999879 |
| /health/liveliness | Passed ✅ | 78 | 79.74354278644388 | 15.375164765794768 | 0.006680497399867377 | 4603 | 2 | 73.53123199999345 | 1243.280717999994 |
| /health/readiness | Passed ✅ | 78 | 79.7319151787119 | 15.251575563897221 | 0.0 | 4566 | 0 | 73.90977500000417 | 1351.1854039999776 |
| Aggregated | Passed ✅ | 78 | 80.47663801829705 | 32.129852244662146 | 0.006680497399867377 | 9619 | 2 | 73.53123199999345 | 1351.1854039999776 |