What's Changed
- Pass router tags in request headers -
x-litellm-tags
+ fix openai metadata param check by @krrishdholakia in #8609 - (Fix) Redis async context usage for Redis Cluster + 94% lower median latency when using Redis Cluster by @ishaan-jaff in #8622
- add openrouter/google/gemini-2.0-flash-001 by @HeMuling in #8619
- feat: add oss license check for related packages by @krrishdholakia in #8623
- fix(model_cost_map): fix json parse error on model cost map + add uni… by @krrishdholakia in #8629
- [Feature]: Redis Caching - Allow setting a namespace for redis cache by @ishaan-jaff in #8624
- Cleanup ui filter icon + pass timeout for Sagemaker messages API by @krrishdholakia in #8630
- Add Elroy to projects built with litellm by @elroy-bot in #8642
- Add OSS license check to ci/cd by @krrishdholakia in #8626
- Fix parallel request limiter on proxy by @krrishdholakia in #8639
- Cleanup user <-> team association on
/team/delete
+ Fix bedrock/deepseek_r1/ translation by @krrishdholakia in #8640 - (Polish/Fixes) - Fixes for Adding Team Specific Models by @ishaan-jaff in #8645
New Contributors
- @HeMuling made their first contribution in #8619
- @elroy-bot made their first contribution in #8642
Full Changelog: v1.61.8-nightly...v1.61.9-nightly
## Docker Run LiteLLM Proxy
```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.9-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Failed ❌ | 180.0 | 203.54644847482734 | 6.3054769799102575 | 6.3054769799102575 | 1887 | 1887 | 146.3379119999786 | 3805.3281139999626 |
Aggregated | Failed ❌ | 180.0 | 203.54644847482734 | 6.3054769799102575 | 6.3054769799102575 | 1887 | 1887 | 146.3379119999786 | 3805.3281139999626 |