What's Changed
- [Docs] use litellm sdk with litellm proxy server by @ishaan-jaff in #5367
- [Feat] Add support for fine tuned vertexai models by @ishaan-jaff in #5371
- [Refactor] Refactor cohere provider to be in a folder by @ishaan-jaff in #5374
- docs: Add pricing for imagen-3 and imagen-3-fast by @ushuz in #5375
- [Feat-Proxy] Allow regenerating proxy virtual keys by @ishaan-jaff in #5379
- [Feat-Proxy] allow regenerating Virtual Keys by @ishaan-jaff in #5378
- build(deps): bump micromatch from 4.0.5 to 4.0.8 in /ui/litellm-dashboard by @dependabot in #5380
Full Changelog: v1.44.6...v1.44.7
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.7
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 130.0 | 153.00695824408174 | 6.357979558515746 | 0.0 | 1901 | 0 | 107.4070600000141 | 1756.0737669999753 |
| Aggregated | Passed ✅ | 130.0 | 153.00695824408174 | 6.357979558515746 | 0.0 | 1901 | 0 | 107.4070600000141 | 1756.0737669999753 |