BerriAI/litellm v1.44.3

on GitHub

🔥 We're launching support for using Bedrock Guardrails on LiteLLM Gateway - use Bedrock guardrails with 100+ LLMs supported by LiteLLM

👉 Start here: https://docs.litellm.ai/docs/proxy/guardrails/bedrock

🔐 Support for using tenant_id, client_id (Entrata ID) auth for Azure OpenAI

https://docs.litellm.ai/docs/providers/azure#usage---litellm-proxy-server

⚡️ [Feat-Proxy] Prometheus Metrics to Track request latency, track llm api latency by

📖 Add example curl on /chat, /completions and /embeddings doc string

📖 Fix doc string for /user/delete

What's Changed

• build(deps): bump hono from 4.2.7 to 4.5.8 in /litellm-js/spend-logs by @dependabot in #5331
• [Feat-Proxy] Prometheus Metrics to Track request latency, track llm api latency by @ishaan-jaff in #5335
• docs(projects): add dbally to sidebar by @micpst in #5336
• feat(caching.py): redis cluster support by @krrishdholakia in #5325
• [Feat-Proxy] add bedrock guardrails support by @ishaan-jaff in #5339
• [Feat] Azure OpenAI add support for using azure_ad_token_provider with LiteLLM Proxy + Router by @ishaan-jaff in #5332

New Contributors

• @micpst made their first contribution in #5336

Full Changelog: v1.44.2...v1.44.3

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.44.3

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results