What's Changed
- [Test] Assert num Callbacks on Proxy don't increase by @ishaan-jaff in #3423
- UI - set DB Exceptions webhook_url on UI by @ishaan-jaff in #3426
- docs - simplify best practices for prod by @ishaan-jaff in #3431
- feat(proxy_server.py): return api base in response headers by @krrishdholakia in #3430
- [Test] - Ensure only 1 slack callback + Size of of all callbacks do not grow by @ishaan-jaff in #3427
- [Test] Add Slack Alerting unit tests by @ishaan-jaff in #3429
- Feat - add bedrock titan embed-v2 by @ishaan-jaff in #3432
- Admin UI - filter exceptions by model group by @ishaan-jaff in #3435
- fix(anthropic.py): drop unsupported non-whitespace character value wh… by @krrishdholakia in #3436
- fix(bedrock.py): convert httpx.timeout to boto3 valid timeout by @krrishdholakia in #3433
Full Changelog: v1.35.37...v1.35.38
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 41 | 50.66770748000066 | 1.6698372244241906 | 0.0 | 500 | 0 | 35.41266199999882 | 1175.5095739999888 |
| /health/liveliness | Passed ✅ | 26 | 28.758076696307747 | 15.285689952379041 | 0.006679348897696762 | 4577 | 2 | 23.39240899999595 | 1262.2757970000293 |
| /health/readiness | Passed ✅ | 26 | 28.897759206984396 | 15.779961770808601 | 0.0 | 4725 | 0 | 23.345480999978463 | 1166.2576729999614 |
| Aggregated | Passed ✅ | 26 | 29.94302010120413 | 32.735488947611834 | 0.006679348897696762 | 9802 | 2 | 23.345480999978463 | 1262.2757970000293 |