🚨 Unstable Release - error with key auth if user not in db detected. Fix being worked on currently. Follow here - #5111
What's Changed
- feat(router.py): allows /chat/completion endpoint to work for request prioritization calls by @krrishdholakia in #5101
- fix(user_api_key_auth.py): respect team budgets over user budget, if key belongs to team by @krrishdholakia in #5099
Full Changelog: v1.43.2...v1.43.3
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.43.3
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 110.0 | 136.2897277365932 | 6.354799579736435 | 6.354799579736435 | 1902 | 1902 | 90.97570500000529 | 2509.1231650000054 |
| Aggregated | Failed ❌ | 110.0 | 136.2897277365932 | 6.354799579736435 | 6.354799579736435 | 1902 | 1902 | 90.97570500000529 | 2509.1231650000054 |