What's Changed
- fix(proxy_server.py): support tracking org spend by @krrishdholakia in #2978
- Fix .dockerignore by @bcvanmeurs in #2982
- fix - stop printing api_key in debug mode by @ishaan-jaff in #2977
- truncate long slack msg by @Dev-Khant in #2984
- [Feat] Support + Test caching for TextCompletion by @ishaan-jaff in #2991
- ui - fix setting slack alerting by @ishaan-jaff in #2985
New Contributors
- @bcvanmeurs made their first contribution in #2982
- @Dev-Khant made their first contribution in #2984
Full Changelog: v1.35.2.dev3...v1.35.2.dev4
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 92 | 99.49399542663635 | 1.4799813646970934 | 0.0 | 443 | 0 | 85.82718999997496 | 1355.355900999939 |
| /health/liveliness | Passed ✅ | 78 | 80.01183602425647 | 15.28757274233386 | 0.003340815721663868 | 4576 | 1 | 73.83316000002083 | 846.8029710000167 |
| /health/readiness | Passed ✅ | 78 | 80.38568712813309 | 15.46129515986038 | 0.0 | 4628 | 0 | 73.86776399999917 | 1050.429261000005 |
| Aggregated | Passed ✅ | 78 | 81.08582581631553 | 32.22884926689133 | 0.003340815721663868 | 9647 | 1 | 73.83316000002083 | 1355.355900999939 |