BerriAI/litellm v1.40.27

on GitHub

✨ Thrilled to launch support for @NVIDIA NIM LLM API on @LiteLLM 1.40.27 👉 Start here: https://docs.litellm.ai/docs/providers/nvidia_nim

🔥 Proxy 100+ LLMS & set budgets

🔑 [Enterprise] Add secret detection pre call hook https://docs.litellm.ai/docs/proxy/enterprise#content-moderation

🛠️ [Fix] - use n in mock completion response on litellm mock responses

⚡️ [Feat] add endpoint to debug memory utilization

🔑 enterprise - allow verifying license in air gapped vpc

What's Changed

• [Fix-Improve] Improve Ollama prompt input and fix Ollama function calling key error and fix Ollama function calling can only join an iterable error by @CorrM in #4373
• Fix Groq Prices by @kiriloman in #4401
• [Feat] add endpoint to debug memory util by @ishaan-jaff in #4364
• [Feat-New Provider] Add Nvidia NIM by @ishaan-jaff in #4403
• [Fix] - use n in mock completion responses by @ishaan-jaff in #4405
• enterprise - allow verifying license in air gapped vpc by @ishaan-jaff in #4409
• Create litellm user to fix issue with prisma in k8s by @lolsborn in #4402
• [Enterprise] Add secret detection pre call hook by @ishaan-jaff in #4410
• Revert "Create litellm user to fix issue with prisma in k8s " by @krrishdholakia in #4412
• fix(router.py): set cooldown_time: per model by @krrishdholakia in #4411

New Contributors

• @CorrM made their first contribution in #4373
• @kiriloman made their first contribution in #4401

Full Changelog: v1.40.26...v1.40.27

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.40.27

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results