What's Changed
- Fix route check for non-proxy admins on jwt auth by @krrishdholakia in #9454
- docs(predibase): fix typo by @luisegarduno in #9464
- build(deps): bump next from 14.2.21 to 14.2.25 in /ui/litellm-dashboard by @dependabot in #9458
- [Feat] Add OpenAI Web Search Tool Call Support - Initial support by @ishaan-jaff in #9465
- Refactor vertex ai passthrough routes - fixes unpredictable behaviour w/ auto-setting default_vertex_region on router model add by @krrishdholakia in #9467
- [Feat] Add testing for
litellm.supports_web_search()and render supports_web_search on model hub by @ishaan-jaff in #9469 - Litellm dev 03 22 2025 release note by @krrishdholakia in #9475
- build: add new vertex text embedding model by @krrishdholakia in #9476
- enables viewing all wildcard models on /model/info by @krrishdholakia in #9473
- Litellm redis semantic caching by @tylerhutcherson in #9356
New Contributors
- @luisegarduno made their first contribution in #9464
Full Changelog: v1.63.14-nightly...1.64.0.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-1.64.0.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 530.0 | 595.2345953515114 | 5.639441403576418 | 0.0 | 1687 | 0 | 488.5956009999859 | 4882.238139000037 |
| Aggregated | Failed ❌ | 530.0 | 595.2345953515114 | 5.639441403576418 | 0.0 | 1687 | 0 | 488.5956009999859 | 4882.238139000037 |