What's Changed
- fix opentelemetry-semantic-conventions-ai does not exist on LiteLLM Docker by @ishaan-jaff in #4129
- [Feat] OTEL - allow propagating traceparent in headers by @ishaan-jaff in #4133
- Added
mypyto the Poetrydevgroup by @jamesbraza in #4136 - Azure AI support all models by @krrishdholakia in #4134
- feat(utils.py): bump tiktoken dependency to 0.7.0 (gpt-4o token counting support) by @krrishdholakia in #4119
- fix(proxy_server.py): use consistent 400-status code error code for exceeded budget errors by @krrishdholakia in #4139
- Allowing inference of LLM provider in
get_supported_openai_paramsby @jamesbraza in #4137 - [FEAT] log management endpoint logs to otel by @ishaan-jaff in #4138
New Contributors
- @jamesbraza made their first contribution in #4136
Full Changelog: v1.40.8...v1.40.9
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.9
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 95 | 118.26463258740928 | 6.42020613574963 | 0.0 | 1922 | 0 | 78.571060999991 | 1634.9082140000064 |
| Aggregated | Passed ✅ | 95 | 118.26463258740928 | 6.42020613574963 | 0.0 | 1922 | 0 | 78.571060999991 | 1634.9082140000064 |