🚨 Stable Release: v1.34.1
What's Changed
- fix for: when using ModelResponse.json() to save and then reconstruct a ModelResponse the choices field ends up empty by @TashaSkyUp in #2656
New Contributors
- @TashaSkyUp made their first contribution in #2656
Full Changelog: v1.34.4...v1.34.4.dev1
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 90 | 93.70739589727417 | 1.5939533105431847 | 0.0 | 477 | 0 | 84.94604300000219 | 596.1977810000008 |
| /health/liveliness | Passed ✅ | 76 | 79.51246170072424 | 15.24113427544542 | 0.0 | 4561 | 0 | 73.50990100002264 | 1350.6406099999992 |
| /health/readiness | Passed ✅ | 76 | 79.16263903972782 | 15.224426169464882 | 0.0 | 4556 | 0 | 73.57008600001791 | 1351.137714999993 |
| Aggregated | Passed ✅ | 76 | 80.05208975661895 | 32.05951375545349 | 0.0 | 9594 | 0 | 73.50990100002264 | 1351.137714999993 |