What's Changed
- Fix #2713 Remove duplicated "blocked" field on LiteLLM_TeamTable by @readevalprint in #2714
- Updating the default Claude3 max tokens by @rmann-nflx in #2701
- (fix) bump uvicorn on proxy docker builds by @ishaan-jaff in #2719
- (fix) Proxy - remove background tasks by @ishaan-jaff in #2721
- feat(auth_checks.py): enable admin to enforce 'user' param for all openai endpoints by @krrishdholakia in #2726
- feat(proxy_server.py): new
/spend/calculateendpoint by @krrishdholakia in #2725 - [FEAT] Improve Proxy Perf - access router model names in constant time by @ishaan-jaff in #2723
- [FEAT] Proxy - reduce deep copies by @ishaan-jaff in #2728
New Contributors
- @readevalprint made their first contribution in #2714
- @rmann-nflx made their first contribution in #2701
Full Changelog: v1.34.6...v1.34.8
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 39 | 47.89723312525114 | 1.6532145452982783 | 0.003339827364238946 | 495 | 1 | 35.51040400003558 | 1220.0857709999582 |
| /health/liveliness | Passed ✅ | 26 | 28.17128257893658 | 15.633731892002507 | 0.006679654728477892 | 4681 | 2 | 23.512639999978546 | 1222.522328000025 |
| /health/readiness | Passed ✅ | 26 | 30.12289661691968 | 15.396604149141542 | 0.0 | 4610 | 0 | 23.55434899999409 | 1311.0719019999806 |
| Aggregated | Passed ✅ | 26 | 30.08843833568385 | 32.68355058644232 | 0.010019482092716837 | 9786 | 3 | 23.512639999978546 | 1311.0719019999806 |