What's Changed
- ui - cleanup showing router settings by @ishaan-jaff in #3249
- (utils.py) - Add seed for Groq by @Manouchehri in #3252
- Add missing tool_calls and name to messages by @elisalimli in #3124
- [Fix] Refactor slack alerting, use one httpx client by @ishaan-jaff in #3255
- fix(utils.py): fix 'no-cache': true when caching is turned on by @krrishdholakia in #3250
- UI - filter metrics by model group by @ishaan-jaff in #3259
Full Changelog: v1.35.20.dev2...v1.35.21
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 58 | 64.9741058238994 | 1.5931582433810807 | 0.0 | 477 | 0 | 51.917337999952906 | 869.7110660000362 |
| /health/liveliness | Passed ✅ | 41 | 44.58716365170725 | 15.554167587894534 | 0.006679908777279165 | 4657 | 2 | 38.56498999999758 | 1391.5798419999987 |
| /health/readiness | Passed ✅ | 41 | 43.37650948992081 | 15.243551829751054 | 0.0 | 4564 | 0 | 38.630514000033145 | 1020.0773849999791 |
| Aggregated | Passed ✅ | 41 | 45.02015455929064 | 32.39087766102667 | 0.006679908777279165 | 9698 | 2 | 38.56498999999758 | 1391.5798419999987 |