Track Cost for Custom Tags
Start here: Live now: https://docs.litellm.ai/docs/proxy/enterprise#tracking-spend-for-custom-tags
👉 /cache/delete live now - delete specific cache keys https://docs.litellm.ai/docs/proxy/caching#deleting-cache-keys---cachedelete
⭐️ Cache Hit Responses include cache-key used in response headers
What's Changed
- fix(proxy_server.py): persist models added via
/model/newto db by @krrishdholakia in #2827 - Fix link to providers in README by @rjmacarthy in #2836
- [FEAT] Proxy - Delete Cache Keys + return cache key in responses by @ishaan-jaff in #2840
- docs -
deletecache keys by @ishaan-jaff in #2846 - [UI] Show Usage Per Tag by @ishaan-jaff in #2850
- feat(ui): add models via ui by @krrishdholakia in #2851
New Contributors
- @rjmacarthy made their first contribution in #2836
Full Changelog: v1.34.25...v1.34.26
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 81 | 90.22493932662181 | 1.4930019313847342 | 0.0 | 447 | 0 | 76.62882199997512 | 1052.6339859999894 |
| /health/liveliness | Passed ✅ | 66 | 69.8399791989995 | 15.35754559397541 | 0.0 | 4598 | 0 | 63.2494559999941 | 1154.2658040000333 |
| /health/readiness | Passed ✅ | 66 | 68.53321177929936 | 15.36088564303891 | 0.0 | 4599 | 0 | 63.40399799995566 | 973.5464270000307 |
| Aggregated | Passed ✅ | 66 | 70.16165628463285 | 32.211433168399054 | 0.0 | 9644 | 0 | 63.2494559999941 | 1154.2658040000333 |