BETA support for Triton Inference Embeddings on 👉 Start here: https://docs.litellm.ai/docs/providers/triton-inference-server
⚡️ [Feat] Use Team based callbacks for failure_callbacks https://docs.litellm.ai/docs/proxy/team_based_routing#logging--caching
🛠️ [Test] Added Testing to ensure Proxy - uses the same OpenAI Client after 1 min
🛠️ [Fix] Upsert deployment bug on LiteLLM Proxy
🔥 Improved LiteLLM-stable load tests - added testing for Azure OpenAI, and using 50+ deployments on a proxy server
🚀 [Feat] support stream_options on litellm.text_completion
What's Changed
- [Fix] Upsert deployment bug by @ishaan-jaff in #3569
- [Test] Proxy - uses the same OpenAI Client after 1 min by @ishaan-jaff in #3570
- [Feat] Use Team based callbacks with litellm.failure_callbacks by @ishaan-jaff in #3573
- enforce unique key and team aliases in the ui by @powerhouseofthecell in #3572
- Huggingface classifier support by @krrishdholakia in #3571
- [Feat] Add Triton Embeddings to LiteLLM by @ishaan-jaff in #3577
- fix(langfuse.py): fix logging user_id in trace param on new trace creation by @krrishdholakia in #3576
Full Changelog: v1.37.2...v1.37.3
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.3
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat