What's Changed
- (feat) Langfuse - Add location logging, and add cache_hit to metadata. by @Manouchehri in #2961
- [FEAT] Add
groq/llama3by @ishaan-jaff in #3176 - [UI] Show teams as dropdown in invite user flow by @ishaan-jaff in #3180
- [FEAT] Log team alias to langfuse by @ishaan-jaff in #3175
- fix: Stream completion responses from anthropic. (Fix #3129) by @jmandel in #3174
- [Fix] - Langfuse log proxy_base_url to langfuse as a tag (if set by user) by @ishaan-jaff in #3183
- fix(caching.py): dual cache async_batch_get_cache fix + testing by @krrishdholakia in #3179
- fix(caching.py): fix redis url parsing logic to work with ssl urls by @krrishdholakia in #3173
New Contributors
Full Changelog: v1.35.15...v1.35.16
Don't want to maintain your internal proxy? get in touch 🎉Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 77 | 85.19056921350604 | 1.5329734178477303 | 0.0 | 459 | 0 | 70.44526700002507 | 1436.744482999984 |
| /health/liveliness | Passed ✅ | 61 | 65.12887882126974 | 15.042510400841344 | 0.0 | 4504 | 0 | 58.82374399999435 | 1409.0413549999994 |
| /health/readiness | Passed ✅ | 61 | 64.03151302740865 | 15.596919087906102 | 0.003339811367859979 | 4670 | 1 | 59.10709400001224 | 1196.9403539999917 |
| Aggregated | Passed ✅ | 61 | 65.55279843433988 | 32.172402906595174 | 0.003339811367859979 | 9633 | 1 | 58.82374399999435 | 1436.744482999984 |