Full Changelog: v1.35.26...v1.35.26.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 94 | 103.60488741425448 | 1.4996401354988889 | 0.0 | 449 | 0 | 86.57567200000926 | 1088.8625389999902 |
| /health/liveliness | Passed ✅ | 78 | 82.48041119317014 | 15.647692727867026 | 0.0 | 4685 | 0 | 73.77187099996263 | 1435.7541210000022 |
| /health/readiness | Passed ✅ | 78 | 81.45476896904056 | 15.10327993925607 | 0.0033399557583494188 | 4522 | 1 | 73.75126800002363 | 1574.0074770000092 |
| Aggregated | Passed ✅ | 78 | 82.98237222110606 | 32.25061280262199 | 0.0033399557583494188 | 9656 | 1 | 73.75126800002363 | 1574.0074770000092 |