What's Changed
- Add token counting for OpenAI tools/tool_choice by @pamelafox in #4716
- [UI] Clear cookies on logging out by @ishaan-jaff in #4743
- ui - allow admin viewer to view caching analytics page by @ishaan-jaff in #4741
- Docs - How to invite users to view usage, caching by @ishaan-jaff in #4744
- [Fix-Proxy] Slack Alerting Store Request Status in memory by @ishaan-jaff in #4732
- feat - use custom api key header name when using litellm virtual keys by @ishaan-jaff in #4739
- fix: enables batch embedding support for triton by @davidschuler-8451 in #4736
- [Fix + Test] anthropic raise litellm.AuthenticationError when no Anthropic API Key provided by @ishaan-jaff in #4747
- fix(proxy/utils.py): fix failure logging for rejected requests by @krrishdholakia in #4742
- fix(factory.py): use stronger typing for anthropic translation by @krrishdholakia in #4746
- ui-fix: clear token when user logging in by @ishaan-jaff in #4755
- [Fix] Langsmith - Don't Log Provider API Keys by @ishaan-jaff in #4754
- [Feat] Proxy + Langsmith - Log user_api_key_user_id, user_api_key_team_alias by @ishaan-jaff in #4757
- fix(litellm_logging.py): don't run async caching for sync streaming calls by @krrishdholakia in #4756
- [Feat] Use Async Httpx client for langsmith logging by @ishaan-jaff in #4758
- add new groq tool models by @themrzmaster in #4751
- Helicone headers to metadata by @maamalama in #4763
- Add Medlm cost calc by @skucherlapati in #4760
- Fix failing tests on PR-4760 by @skucherlapati in #4765
New Contributors
- @pamelafox made their first contribution in #4716
- @davidschuler-8451 made their first contribution in #4736
- @skucherlapati made their first contribution in #4760
Full Changelog: v1.41.23...v1.41.24
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.41.24
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 100.0 | 129.16254622602372 | 6.444436535071256 | 0.0 | 1929 | 0 | 86.27257299997382 | 2671.388750999995 |
Aggregated | Passed ✅ | 100.0 | 129.16254622602372 | 6.444436535071256 | 0.0 | 1929 | 0 | 86.27257299997382 | 2671.388750999995 |