What's Changed
- Feat - add groq tool calling + testing by @ishaan-jaff in #3029
- Docs - add groq tool calling example by @ishaan-jaff in #3030
- fix(anthropic_text.py): add support for async text completion calls by @krrishdholakia in #3028
- [Fix] show users tab on user + fix pagination by @ishaan-jaff in #3033
- Docs - add team based logging langfuse on langfuse proxy docs by @ishaan-jaff in #3041
- [Fix + Test] - Team Based logging to Langfuse by @ishaan-jaff in #3039
- [fix] using wildcard openai models proxy by @ishaan-jaff in #3042
- fix(vertex_ai_anthropic.py): Add vertex ai models via
/model/newby @krrishdholakia in #3043 - fix(proxy_server.py): fix /team/update endpoint by @krrishdholakia in #3034
Full Changelog: v1.35.5...v1.35.6
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 80 | 91.32192700686522 | 1.4596854141948363 | 0.0033402412224138134 | 437 | 1 | 75.3214950000256 | 1438.5441430000014 |
| /health/liveliness | Passed ✅ | 66 | 68.08952963725082 | 15.405192517772507 | 0.0 | 4612 | 0 | 63.32589900000585 | 1327.5218320000022 |
| /health/readiness | Passed ✅ | 66 | 68.26662181489549 | 15.338387693324231 | 0.0033402412224138134 | 4592 | 1 | 46.533456000020124 | 1448.8824260000115 |
| Aggregated | Passed ✅ | 66 | 69.22693913110682 | 32.20326562529157 | 0.006680482444827627 | 9641 | 2 | 46.533456000020124 | 1448.8824260000115 |