What's Changed
- [Fix] OTEL - Unsupported | type annotations in python3.9 by @ishaan-jaff in #5548
- Update Hugging Face Doc by @gary149 in #5411
- [Fix-Datdog Logger] Log exceptions when callbacks faces an error by @ishaan-jaff in #5570
- fix(navbar.tsx): only show 'get enterprise license' if user is not already a premium user by @krrishdholakia in #5568
- LiteLLM Minor Fixes and Improvements (08/06/2024) by @krrishdholakia in #5567
- [Feat-Proxy] Use DB Views to Get spend per Tag (Usage endpoints) by @ishaan-jaff in #5574
- [Feat] Allow setting duration time when regenerating key by @ishaan-jaff in #5566
- [Feat] Add cost tracking for cohere rerank by @ishaan-jaff in #5573
- Allow client-side credentials to be sent to proxy (accept only if complete credentials are given) by @krrishdholakia in #5575
New Contributors
Full Changelog: v1.44.19...v1.44.21-stable
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.21-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 110.0 | 137.28568682202445 | 6.405426266891142 | 0.0 | 1916 | 0 | 96.66716199996017 | 1538.306079999984 |
| Aggregated | Passed ✅ | 110.0 | 137.28568682202445 | 6.405426266891142 | 0.0 | 1916 | 0 | 96.66716199996017 | 1538.306079999984 |