What's Changed
- fix(bedrock.py): support bedrock claude 3 function calling when stream=true by @krrishdholakia in #2630
- (fix) include tenacity in req.txt by @ishaan-jaff in #2619
- [ fix ] retry logic - when using router/proxy - don't retry on the litellm.completion level too by @ishaan-jaff in #2620
- Bump fastapi version
0.104.1to0.109.1by @RoniGurvich in #2617
New Contributors
- @RoniGurvich made their first contribution in #2617
Full Changelog: v1.33.2...v1.33.3
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 98 | 106.21492467973763 | 1.5333765656063003 | 0.0 | 459 | 0 | 94.59735499996214 | 1056.5959150000026 |
| /health/liveliness | Passed ✅ | 79 | 84.34175220631124 | 15.350469104490088 | 0.0 | 4595 | 0 | 76.97201599995651 | 7435.313418000022 |
| /health/readiness | Passed ✅ | 79 | 85.00047014053042 | 14.976311859723408 | 0.0 | 4483 | 0 | 77.01827100004266 | 7457.214043999955 |
| Aggregated | Passed ✅ | 79 | 85.704111298731 | 31.860157529819798 | 0.0 | 9537 | 0 | 76.97201599995651 | 7457.214043999955 |