Excited to launch major improvement to LiteLLM UI - Create Teams, Keys and Track LLM Spend
📈 New /metrics endpoint for viewing metrics on Prometheus: https://docs.litellm.ai/docs/proxy/grafana_metrics
Admin UI Upgrades:
⚡️ Test Keys on UI (make an LLM Request)
🚀 Delete Teams
⭐️ Add all_models to Teams, Keys
What's Changed
- [Fix + Docs] grafana / prometheus metrics by @ishaan-jaff in #2752
- [UI] /team/delete by @ishaan-jaff in #2759
- build(deps): bump express from 4.18.2 to 4.19.2 in /docs/my-website by @dependabot in #2689
- build(deps): bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /docs/my-website by @dependabot in #2628
- [UI] team, key add
all_modelsby @ishaan-jaff in #2760
Full Changelog: v1.34.13...v1.34.14
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 1800.0 | 1760.7637035391895 | 1.4060758597299565 | 1.4060758597299565 | 421 | 421 | 342.6113250000071 | 3130.634011999973 |
| /health/liveliness | Passed ✅ | 37 | 39.48335569165602 | 14.969197157505143 | 0.0 | 4482 | 0 | 35.50325700001622 | 1117.3830359999783 |
| /health/readiness | Passed ✅ | 37 | 40.18390747290733 | 15.162908321078392 | 0.0033398476478146235 | 4540 | 1 | 35.52504999998973 | 1222.5566220000132 |
| Aggregated | Passed ✅ | 37 | 116.56050612379507 | 31.538181338313493 | 1.409415707377771 | 9443 | 422 | 35.50325700001622 | 3130.634011999973 |