What's Changed
- [Fix] OTEL - Unsupported | type annotations in python3.9 by @ishaan-jaff in #5548
- Update Hugging Face Doc by @gary149 in #5411
- [Fix-Datdog Logger] Log exceptions when callbacks faces an error by @ishaan-jaff in #5570
- fix(navbar.tsx): only show 'get enterprise license' if user is not already a premium user by @krrishdholakia in #5568
- LiteLLM Minor Fixes and Improvements (08/06/2024) by @krrishdholakia in #5567
- [Feat-Proxy] Use DB Views to Get spend per Tag (Usage endpoints) by @ishaan-jaff in #5574
- [Feat] Allow setting duration time when regenerating key by @ishaan-jaff in #5566
- [Feat] Add cost tracking for cohere rerank by @ishaan-jaff in #5573
- Allow client-side credentials to be sent to proxy (accept only if complete credentials are given) by @krrishdholakia in #5575
New Contributors
Full Changelog: v1.44.19...v1.44.21
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.21
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 130.0 | 166.20606541064828 | 6.339987548576783 | 0.0 | 1897 | 0 | 112.93849100002262 | 2116.0177159999876 |
| Aggregated | Passed ✅ | 130.0 | 166.20606541064828 | 6.339987548576783 | 0.0 | 1897 | 0 | 112.93849100002262 | 2116.0177159999876 |