🚨 'simple-shuffle' async migration, missing random + tpm shuffle. Added in v1.35.20.
What's Changed
- fix(_redis.py): support redis ssl as a kwarg
REDIS_SSLby @krrishdholakia in #3191 - FIX: ollama chat completion proxy internal server 500 by @merefield in #3189
- Disable special tokens in ollama completion when counting tokens by @rick-github in #3170
- [Fix] -
/audio/transcriptionssecurity fix by @ishaan-jaff in #3193 - [UI] - non admin flow - only Create + Test Key available by @ishaan-jaff in #3184
- Load google ADC before init AnthropicVertex by @ushuz in #3150
- Fix tool call errors using anthropic by @n1lanjan in #3118
- Fix new line issue in cohere_message_pt by @elisalimli in #3115
- fix - slack alerting show
inputfor embedding requests by @ishaan-jaff in #3196 - fix(router.py): Make TPM limits concurrency-safe by @krrishdholakia in #3192
- [UI] - simplify "Create Key" for non admins by @ishaan-jaff in #3197
- ui - fix create key flow / cleanup non admin flow by @ishaan-jaff in #3198
- Revert "Load google ADC before init AnthropicVertex" by @ishaan-jaff in #3203
New Contributors
- @merefield made their first contribution in #3189
- @rick-github made their first contribution in #3170
- @n1lanjan made their first contribution in #3118
- @elisalimli made their first contribution in #3115
Full Changelog: v1.35.17...v1.35.18
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 81 | 84.52471885930657 | 1.5432420439247365 | 0.0 | 462 | 0 | 75.13792499997862 | 512.1452240000224 |
| /health/liveliness | Passed ✅ | 66 | 67.80148838042581 | 15.085023961827078 | 0.0 | 4516 | 0 | 63.41557500002182 | 575.8615130000067 |
| /health/readiness | Passed ✅ | 66 | 68.58760093607881 | 15.57271517051325 | 0.0 | 4662 | 0 | 63.40643299995463 | 1172.486718000016 |
| Aggregated | Passed ✅ | 66 | 68.98312626587143 | 32.20098117626507 | 0.0 | 9640 | 0 | 63.40643299995463 | 1172.486718000016 |