What's Changed
- Add datadog health check support + fix bedrock converse cost tracking w/ region name specified by @krrishdholakia in #7958
- Retry for replicate completion response of status=processing (#7901) by @krrishdholakia in #7965
- Ollama ssl verify = False + Spend Logs reliability fixes by @krrishdholakia in #7931
- (Feat) - allow setting
default_onguardrails by @ishaan-jaff in #7973 - (Testing) e2e testing for team budget enforcement checks by @ishaan-jaff in #7988
- (UI) - Usage page show days when spend is 0 and round spend figures on charts to 2 sig figs by @ishaan-jaff in #7991
- (Feat) - Add GCS Pub/Sub Logging integration for sending DB
SpendLogsto BigQuery by @ishaan-jaff in #7976 - fix(spend_tracking_utils.py): revert api key pass through fix by @krrishdholakia in #7977
- Ensure base_model cost tracking works across all endpoints by @krrishdholakia in #7989
- (UI) Allow admin to expose teams for joining by @krrishdholakia in #7992
Full Changelog: v1.59.6...v1.59.7
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.7
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 260.0 | 294.5630730660492 | 6.1254059494010225 | 0.0 | 1832 | 0 | 231.04980300001898 | 2728.9633709999634 |
| Aggregated | Passed ✅ | 260.0 | 294.5630730660492 | 6.1254059494010225 | 0.0 | 1832 | 0 | 231.04980300001898 | 2728.9633709999634 |