What's Changed
- feat(langfuse.py): Allow for individual call message/response redaction by @alexanderepstein in #3603
- [Feat] -
/global/spend/reportby @ishaan-jaff in #3619 - Fixes #3544 based on the data-type of message by @paneru-rajan in #3554
- [UI] Filter Tag Spend by Date + Show Bar Chart by @ishaan-jaff in #3624
- Default routing fallbacks by @krrishdholakia in #3625
Full Changelog: v1.37.7...v1.37.9
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.9
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 40 | 45.14681571397024 | 1.5067595942578198 | 1.5067595942578198 | 451 | 451 | 37.28894399998239 | 203.69157899997958 |
| /health/liveliness | Failed ❌ | 38 | 43.774724098143416 | 15.65894061704302 | 15.65894061704302 | 4687 | 4687 | 36.20009499996968 | 219.30193999997982 |
| /health/readiness | Failed ❌ | 38 | 42.98829494917115 | 15.314824789529593 | 15.314824789529593 | 4584 | 4584 | 36.154727999985425 | 234.44879100000549 |
| Aggregated | Failed ❌ | 38 | 43.46756735054526 | 32.48052500083043 | 32.48052500083043 | 9722 | 9722 | 36.154727999985425 | 234.44879100000549 |