What's Changed
- Fix Anthropic Messages Prompt Template function to add a third condition: list of text-content dictionaries by @eercanayar in #2780
- add safety_settings parameters to Vertex AI async mode by @hellof20 in #3312
- (feat) add IBM watsonx.ai as an llm provider by @simonsanvil in #3270
- fix: incorrect analysis for vertex ai env vars causing confusion by @suptejas in #3324
- Revert "Fix Anthropic Messages Prompt Template function to add a third condition: list of text-content dictionaries" by @krrishdholakia in #3328
- Add watsonx to list of model providers and fixed typo in colab notebook by @simonsanvil in #3326
- fix(router.py): fix default retry logic by @krrishdholakia in #3302
- [Feat] Redact Logging Messages/Response content on Logging Providers with
litellm.turn_off_message_logging=Trueby @ishaan-jaff in #3330 - Fix - Admin UI stops working if proxy budget has been exceeded by @ishaan-jaff in #3335
- docs - alerting by @ishaan-jaff in #3337
- UI - fix bug showing models to pic by @ishaan-jaff in #3339
- [Fix] - Link Langfuse Traces to Slack Alerts by @ishaan-jaff in #3336
- Fix - slack alerting show deployment latencies in sorted order by @ishaan-jaff in #3338
New Contributors
- @eercanayar made their first contribution in #2780
- @hellof20 made their first contribution in #3312
- @simonsanvil made their first contribution in #3270
- @suptejas made their first contribution in #3324
Full Changelog: v1.35.30...v1.35.31
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 40 | 52.40982617634495 | 1.6101070107145294 | 0.0 | 482 | 0 | 34.925257999987025 | 1028.7042879999717 |
| /health/liveliness | Passed ✅ | 25 | 27.778927652744667 | 15.75700159655692 | 0.003340470976586161 | 4717 | 1 | 23.23458099999698 | 1168.9134119999949 |
| /health/readiness | Passed ✅ | 25 | 27.787623363912648 | 15.402911673038787 | 0.0 | 4611 | 0 | 23.29059499993491 | 957.4790820000203 |
| Aggregated | Passed ✅ | 25 | 28.993218071967036 | 32.77002028031024 | 0.003340470976586161 | 9810 | 1 | 23.23458099999698 | 1168.9134119999949 |