What's Changed
- fix(utils.py): fix fix pydantic obj to schema creation for vertex en… by @krrishdholakia in #6071
- Proxy: include customer budget in responses by @kvadros in #5977
- (proxy ui) - fix view user pagination by @ishaan-jaff in #6094
- (proxy ui sso flow) - fix invite user sso flow by @ishaan-jaff in #6093
- (bug fix) TTL not being set for embedding caching requests by @ishaan-jaff in #6095
- (feat proxy) add v2 maintained LiteLLM grafana dashboard by @ishaan-jaff in #6098
New Contributors
Full Changelog: v1.48.17...v1.48.18
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.48.18
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 160.0 | 187.4070346405257 | 6.354960841056153 | 0.0 | 1900 | 0 | 124.29662199997438 | 3379.0254470000036 |
| Aggregated | Passed ✅ | 160.0 | 187.4070346405257 | 6.354960841056153 | 0.0 | 1900 | 0 | 124.29662199997438 | 3379.0254470000036 |