What's Changed
- feat(proxy/utils.py): enable updating db in a separate server by @krrishdholakia in #2722
- fix(proxy_server.py): don't auto-create user when creating key by @krrishdholakia in #2724
- Batch embedding for Ollama by @onukura in #2720
- Add
trace_namein Langfuse logging by @andreaponti5 in #2715 - Admin UI clearly show models by @ishaan-jaff in #2735
- [Admin UI] Use consistent spacing, show mandatory fields by @ishaan-jaff in #2738
- (fix) ui - clean up username display by @ishaan-jaff in #2739
- (ui) new build by @ishaan-jaff in #2740
- fix(proxy_server.py): fix tpm/rpm limiting for jwt auth by @krrishdholakia in #2741
New Contributors
- @andreaponti5 made their first contribution in #2715
Full Changelog: v1.34.10...v1.34.12
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 1800.0 | 1755.1105577409173 | 1.379557015059167 | 1.379557015059167 | 413 | 413 | 314.5321039999658 | 3183.5194690000035 |
| /health/liveliness | Passed ✅ | 25 | 26.838340247288826 | 15.398929393275447 | 0.0 | 4610 | 0 | 22.767044999994823 | 1018.2174079999982 |
| /health/readiness | Passed ✅ | 25 | 28.248346697941912 | 15.094959203516646 | 0.0 | 4519 | 0 | 23.172290999980305 | 1362.1405929999924 |
| Aggregated | Passed ✅ | 25 | 102.30975556644304 | 31.87344561185126 | 1.379557015059167 | 9542 | 413 | 22.767044999994823 | 3183.5194690000035 |