What's Changed
- Ensure prompt injection attack 'known phrases' are >= 3 words by @krrishdholakia in #2611
- fix(handle_jwt.py): track spend for user using jwt auth by @krrishdholakia in #2606
- (docs) add example using vertex ai on litellm proxy by @ishaan-jaff in #2612
- (docs) Litellm fix quick start docker by @ishaan-jaff in #2613
- Update proxy_server.py by @eltociear in #2563
- feat(proxy_server.py): enable llm api based prompt injection checks by @krrishdholakia in #2614
Full Changelog: v1.32.9...v1.33.0
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 45 | 51.133834150226775 | 1.4896201673972045 | 0.0 | 446 | 0 | 40.35980700007258 | 1115.5469500000095 |
| /health/liveliness | Passed ✅ | 25 | 27.634887937034062 | 15.7011309572517 | 0.0 | 4701 | 0 | 23.182082000005266 | 1054.0938130000086 |
| /health/readiness | Passed ✅ | 25 | 28.51761090139341 | 15.343755715297661 | 0.0033399555322807272 | 4594 | 1 | 23.214589000019714 | 1252.2123720000309 |
| Aggregated | Passed ✅ | 25 | 29.127112483728485 | 32.534506839946566 | 0.0033399555322807272 | 9741 | 1 | 23.182082000005266 | 1252.2123720000309 |