What's Changed
- (feat) add bedrock/stability.stable-image-ultra-v1:0 by @ishaan-jaff in #6723
- [Feature]: Stop swallowing up AzureOpenAi exception responses in litellm's implementation for a BadRequestError by @ishaan-jaff in #6745
- [Feature]: json_schema in response support for Anthropic by @ishaan-jaff in #6748
- fix: import audio check by @IamRash-7 in #6740
- (fix) Cost tracking for
vertex_ai/imagen3by @ishaan-jaff in #6752 - (feat) Vertex AI - add support for fine tuned embedding models by @ishaan-jaff in #6749
- LiteLLM Minor Fixes & Improvements (11/13/2024) by @krrishdholakia in #6729
- feat - add us.llama 3.1 models by @ishaan-jaff in #6760
New Contributors
- @IamRash-7 made their first contribution in #6740
Full Changelog: v1.52.8...v1.52.8.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.8.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 200.0 | 224.53181540392876 | 6.2943081785528925 | 0.0 | 1884 | 0 | 181.30709700000125 | 3153.26821299999 |
| Aggregated | Passed ✅ | 200.0 | 224.53181540392876 | 6.2943081785528925 | 0.0 | 1884 | 0 | 181.30709700000125 | 3153.26821299999 |