What's Changed
- (Feat) - Allow calling Nova models on
/bedrock/invoke/
by @ishaan-jaff in #8397 - Run litellm in dev mode by @CakeCrusher in #8404
- (Bug Fix) - Bedrock completions with aws_region_name by @ishaan-jaff in #8384
- added gemini 2.0 models to docs by @mubashir1osmani in #8412
- Added filter in Teams and fixed spacing & height issues in Teams tabs (#6192) by @tahaali-dev in #8357
- Revert "Added filter in Teams and fixed spacing & height issues in Teams tabs (#6192)" by @krrishdholakia in #8416
- Allow editing model api key + provider on UI by @krrishdholakia in #8406
- Allow org admin to create teams on UI by @krrishdholakia in #8407
- Azure api version check - fix str compare to convert to int by @krrishdholakia in #8438
- Fix callback add when user_config passed + support passing openai org client-side by @krrishdholakia in #8443
- Org UI Improvements by @krrishdholakia in #8436
- (e2e testing) - add tests for using litellm
/team/
updates in multi-instance deployments with Redis by @ishaan-jaff in #8440 - (Feat) - Allow viewing Request/Response Logs stored in GCS Bucket by @ishaan-jaff in #8449
New Contributors
- @CakeCrusher made their first contribution in #8404
Full Changelog: v1.60.8...v1.61.0
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.0
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 180.0 | 213.86169773089247 | 6.297834462789351 | 0.003342799608699231 | 1884 | 1 | 81.07622899996159 | 4173.802059999957 |
Aggregated | Passed ✅ | 180.0 | 213.86169773089247 | 6.297834462789351 | 0.003342799608699231 | 1884 | 1 | 81.07622899996159 | 4173.802059999957 |