What's Changed
-
Update support for langfuse metadata by @alexanderepstein in #3459
-
Synced the doc with the Mistral by @paneru-rajan in #3471
-
update langchain documentation to reflect refactor by @sepiatone in #3464
-
Add devcontainer by @Manouchehri in #3494
-
Added support for JWT auth with PEM cert public keys by @ghaemisr in #3500
-
[Feat + Test] Add lowest cost routing - litellm.Router by @ishaan-jaff in #3504
https://docs.litellm.ai/docs/routing#advanced---routing-strategies -
[Feat] Make lowest cost routing Async by @ishaan-jaff in #3510
-
feat(slack_alerting.py): reintegrate langfuse trace url for slack alerts by @krrishdholakia in #3506
-
Added "deepseek/" as a supported provider (openai compatible) by @paul-gauthier in #3503
-
[Feat] litellm.Router / litellm.completion - send llm exceptions to slack by @ishaan-jaff in #3511
https://docs.litellm.ai/docs/routing#alerting-
New Contributors
- @alexanderepstein made their first contribution in #3459
- @sepiatone made their first contribution in #3464
- @ghaemisr made their first contribution in #3500
- @mbektas made their first contribution in #3470
Full Changelog: v1.36.1...v1.36.2
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 92 | 96.04003103837316 | 1.4796974346523213 | 0.0 | 443 | 0 | 85.96445500000982 | 442.37826399998426 |
| /health/liveliness | Passed ✅ | 78 | 79.00126169393944 | 15.431607557773644 | 0.0 | 4620 | 0 | 73.91568399998505 | 315.2042289999599 |
| /health/readiness | Passed ✅ | 78 | 81.64751764383558 | 15.117631126944485 | 0.0033401747960549013 | 4526 | 1 | 73.83376600000702 | 1535.723929999989 |
| Aggregated | Passed ✅ | 78 | 81.03746247074821 | 32.02893611937045 | 0.0033401747960549013 | 9589 | 1 | 73.83376600000702 | 1535.723929999989 |