BerriAI/litellm v1.48.11-stable

on GitHub

What's Changed

• (azure): Enable stream_options for Azure OpenAI. (#6024) by @krrishdholakia in #6029
• (load testing) add vertex_ai embeddings load test by @ishaan-jaff in #6004
• (feat proxy) add key based logging for GCS bucket by @ishaan-jaff in #6031
• (feat) add nvidia nim embeddings by @ishaan-jaff in #6032
• (contributor PRs) oct 3rd, 2024 by @ishaan-jaff in #6034
• fix(factory.py): bedrock: merge consecutive tool + user messages by @krrishdholakia in #6028
• (feat) openai prompt caching (non streaming) - add prompt_tokens_details in usage response by @ishaan-jaff in #6039

Full Changelog: v1.48.10...v1.48.11-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.48.11-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.48.11-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.48.11-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.48.11-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results