What's Changed
- docs - add algolia search 🫡 by @ishaan-jaff in #4320
- [Feat] allow using custom router strategy by @ishaan-jaff in #4318
- fix(utils.py): allow dropping specific openai params by @krrishdholakia in #4313
- fix(user_api_key_auth.py): ensure user has access to fallback models by @krrishdholakia in #4321
- Update proxy_cli.py by @vanpelt in #4325
- fix(key_management_endpoints.py): use common _duration_in_seconds function by @krrishdholakia in #4323
- feat(router.py): allow user to call specific deployment via id by @krrishdholakia in #4290
- test(test_python_38.py): add coverage for non-gen settings config.yaml flow by @krrishdholakia in #4328
- [Fix] user field and user_api_key_* is sometimes omitted randomly by @ishaan-jaff in #4322
Full Changelog: v1.40.19...v1.40.20
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.20
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 79 | 105.64488981589672 | 6.515118790194818 | 0.0 | 1950 | 0 | 67.60507300003837 | 3342.9461570000285 |
| Aggregated | Passed ✅ | 79 | 105.64488981589672 | 6.515118790194818 | 0.0 | 1950 | 0 | 67.60507300003837 | 3342.9461570000285 |