What's Changed
- LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
- Litellm dev 11 02 2024 by @krrishdholakia in #6561
- build(deps): bump cookie and express in /docs/my-website by @dependabot in #6566
- docs(virtual_keys.md): update Dockerfile reference by @emmanuel-ferdman in #6554
- (proxy fix) - call connect on prisma client when running setup by @ishaan-jaff in #6534
- Add 3.5 haiku by @paul-gauthier in #6588
- Litellm perf improvements 3 by @krrishdholakia in #6573
- (fix) /image/generation - ImageObject conversion when
content_filter_results
exists by @ishaan-jaff in #6584 - (fix) litellm.text_completion raises a non-blocking error on simple usage by @ishaan-jaff in #6546
- (feat) add
Predicted Outputs
for OpenAI by @ishaan-jaff in #6594 - (fix) Vertex Improve Performance when using
image_url
by @ishaan-jaff in #6593 - fix(lowest_tpm_rpm_routing.py): fix parallel rate limit check by @krrishdholakia in #6577
New Contributors
- @emmanuel-ferdman made their first contribution in #6554
Full Changelog: v1.51.3...v1.52.0
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 210.0 | 231.0704959909717 | 6.291122309918775 | 0.0 | 1883 | 0 | 180.74613400000317 | 2618.3897060000163 |
Aggregated | Passed ✅ | 210.0 | 231.0704959909717 | 6.291122309918775 | 0.0 | 1883 | 0 | 180.74613400000317 | 2618.3897060000163 |