What's Changed
- [Docs+Fixes] Litellm helm chart use k8 1.21 by @ishaan-jaff in #2544
- docs(langfuse): add chatlitellm section by @udit-001 in #2541
- 89% Caching improvement - Async Redis completion calls + batch redis GET requests for a given key + call type by @krrishdholakia in #2542
New Contributors
Full Changelog: v1.31.15...v1.31.16
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 89 | 99.65921707220798 | 1.52648108721305 | 0.0 | 457 | 0 | 81.2413830000196 | 1340.1977020000118 |
| /health/liveliness | Passed ✅ | 62 | 65.55824947976126 | 15.101140033457767 | 0.003340221197402735 | 4521 | 1 | 59.555162999970435 | 1363.859160000004 |
| /health/readiness | Passed ✅ | 190.0 | 185.8658501946554 | 15.37503817164479 | 0.010020663592208207 | 4603 | 3 | 124.73937800001522 | 1249.4980939999891 |
| Aggregated | Passed ✅ | 100 | 124.98419961861957 | 32.002659292315606 | 0.01336088478961094 | 9581 | 4 | 59.555162999970435 | 1363.859160000004 |