What's Changed
- Add JSON mode to Gemini (Vertex AI) by @Manouchehri in #2964
- fix - delete key from inMemory Cache after /key/update by @ishaan-jaff in #2965
- fix(handle_jwt.py): User cost tracking via JWT Auth by @krrishdholakia in #2970
- fix(proxy_server.py): support tracking org spend by @krrishdholakia in #2978
Full Changelog: v1.35.1.dev2...v1.35.2
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 41 | 44.1075405043854 | 1.523085528016257 | 0.0 | 456 | 0 | 35.037465000016255 | 749.7959620000074 |
| /health/liveliness | Passed ✅ | 25 | 27.538079022664157 | 15.474682568638856 | 0.0 | 4633 | 0 | 23.358772999984012 | 1158.9303129999848 |
| /health/readiness | Passed ✅ | 25 | 27.88288945533811 | 15.70514945774658 | 0.0 | 4702 | 0 | 23.31466300000784 | 1296.1237730000335 |
| Aggregated | Passed ✅ | 25 | 28.475365621591475 | 32.702917554401694 | 0.0 | 9791 | 0 | 23.31466300000784 | 1296.1237730000335 |