BerriAI/litellm v1.40.8

on GitHub

What's Changed

• [FEAT]- OTEL log litellm request / response by @ishaan-jaff in #4076
• [Feat] Enterprise - Attribute Management changes to Users in Audit Logs by @ishaan-jaff in #4083
• [FEAT]- OTEL Log raw LLM request/response on OTEL by @ishaan-jaff in #4078
• fix(cost_calculator.py): fixes tgai unmapped model pricing by @krrishdholakia in #4085
• fix(utils.py): improved predibase exception mapping by @krrishdholakia in #4080
• [Fix] Litellm sdk - allow ChatCompletionMessageToolCall, and Function to be used as dict by @ishaan-jaff in #4086
• Update together ai pricing by @krrishdholakia in #4087
• [Feature]: Proxy: Support API-Key header in addition to Authorization header by @ishaan-jaff in #4088
• docs - cache controls on litellm python SDK by @ishaan-jaff in #4099
• docs: add llmcord.py to side bar nav by @jakobdylanc in #4101
• docs: fix llmcord.py side bar link by @jakobdylanc in #4104
• [FEAT] - viewing spend report per customer / team by @ishaan-jaff in #4105
• feat - log Proxy Server auth errors on OTEL by @ishaan-jaff in #4103
• [Feat] Client Side Fallbacks by @ishaan-jaff in #4107
• Fix typos: Enterpise -> Enterprise by @msabramo in #4110
• assistants.md: Remove extra trailing backslash by @msabramo in #4112
• assistants.md: Add "Get a Thread" example by @msabramo in #4114
• ui - Fix Test Key dropdown by @ishaan-jaff in #4108
• fix(bedrock_httpx.py): fix tool calling for anthropic bedrock calls w/ streaming by @krrishdholakia in #4106
• fix(proxy_server.py): allow passing in a list of team members by @krrishdholakia in #4084
• fix - show model group in Azure ContentPolicy exceptions by @ishaan-jaff in #4116

Client Side Fallbacks: https://docs.litellm.ai/docs/proxy/reliability#test---client-side-fallbacks

Full Changelog: v1.40.7...v1.40.8

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.8

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.8

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results