What's Changed
- Add
attempted-retriesandtimeoutvalues to response headers + more testing by @krrishdholakia in #7926 - Refactor prometheus e2e test by @yujonglee in #7919
- (Testing + Refactor) - Unit testing for team and virtual key budget checks by @ishaan-jaff in #7945
- docs: fix typo by @wagnerjt in #7953
- (Feat) - Allow Admin UI users to view spend logs even when not storing messages / responses by @ishaan-jaff in #7952
- (UI) - Set/edit guardrails on a virtual key by @ishaan-jaff in #7954
- (Feat) - emit
litellm_team_budget_reset_at_metricandlitellm_api_key_budget_remaining_hours_metricon prometheus by @ishaan-jaff in #7946 - (Feat) allow setting guardrails on a team on the API by @ishaan-jaff in #7959
- (UI) Set guardrails on Team Create and Edit page by @ishaan-jaff in #7963
- (GCS fix) - don't truncate payload by @ishaan-jaff in #7964
- Litellm dev 01 23 2025 p2 by @krrishdholakia in #7962
New Contributors
Full Changelog: v1.59.5...v1.59.6
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.6
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 250.0 | 302.94444351157557 | 6.065526445072595 | 0.0 | 1814 | 0 | 184.99327999995785 | 3192.1896389999915 |
| Aggregated | Failed ❌ | 250.0 | 302.94444351157557 | 6.065526445072595 | 0.0 | 1814 | 0 | 184.99327999995785 | 3192.1896389999915 |