What's Changed
- Update prices/context windows for Perplexity Llama 3.1 models by @bachya in #5206
- Allow specifying langfuse project for logging in key metadata by @krrishdholakia in #5176
- vertex_ai/claude-3-5-sonnet@20240620 support prefill by @paul-gauthier in #5203
- Enable follow redirects in ollama_chat by @fabceolin in #5148
- feat(user_api_key_auth.py): support calling langfuse with litellm user_api_key_auth by @krrishdholakia in #5192
- Use
AZURE_API_VERSIONenv var as default azure openai version by @msabramo in #5211 - [Feat] Add Anthropic API Prompt Caching Support by @ishaan-jaff in #5210
New Contributors
- @fabceolin made their first contribution in #5148
Full Changelog: v1.43.12...v1.43.13-stable
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.43.13-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 110.0 | 123.61768599690367 | 6.473189780476622 | 0.0 | 1937 | 0 | 85.34288399999923 | 1587.6906680000502 |
| Aggregated | Passed ✅ | 110.0 | 123.61768599690367 | 6.473189780476622 | 0.0 | 1937 | 0 | 85.34288399999923 | 1587.6906680000502 |