What's Changed
- (feat) bump to python 3.11 - Improve Proxy perf 7% by @ishaan-jaff in #2646
- feat(handle_jwt.py): enable jwt-project based auth by @krrishdholakia in #2650
- [FEAT] Add example Kubernetes + Service YAML Files by @ishaan-jaff in #2653
- (feat) remove litellm.telemetry - improve perf on EC2 tiny machines by 90% by @ishaan-jaff in #2657
Full Changelog: v1.33.4...v1.33.7
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 79 | 90.82509035211159 | 1.422902757757835 | 0.0033401473186803637 | 426 | 1 | 68.13304000002063 | 1320.544376999976 |
| /health/liveliness | Passed ✅ | 65 | 67.60595521311457 | 15.484922969402167 | 0.0 | 4636 | 0 | 63.423195999973814 | 1027.6299029999905 |
| /health/readiness | Passed ✅ | 65 | 67.28953461176305 | 15.391398844479117 | 0.0 | 4608 | 0 | 63.412283999980446 | 1321.834434999971 |
| Aggregated | Passed ✅ | 65 | 68.47806332461236 | 32.299224571639115 | 0.0033401473186803637 | 9670 | 1 | 63.412283999980446 | 1321.834434999971 |