BerriAI/litellm v1.74.14-nightly

on GitHub

What's Changed

• [Bug Fix] Pass through logging handler VertexAI - ensure multimodal embedding responses are logged by @ishaan-jaff in #13050
• Remove duplicate test case verifying field filtering logic by @emmanuel-ferdman in #13023
• Properly parse json options for key generation in the UI by @5731la in #12989
• fix: correct CompletionRequest messages type to match OpenAI API spec by @restato in #12980
• chore: Improve docs for cost tracking by @RichardoC in #12976
• feat(langfuse-otel): Add comprehensive metadata support to Langfuse OpenTelemetry integration by @strickvl in #12956
• [MCP Gateway] MCP tools fix scrolling issue by @jugaldb in #13015
• [MCP gateway] add url namespacing docs by @jugaldb in #13063
• [Bug Fix] The model gemini-2.5-flash with the merge_reasoning_content_in_choices parameter does not work by @ishaan-jaff in #13066
• [Feat] Add Google AI Studio Imagen4 model family by @ishaan-jaff in #13065
• Default Usage Chart Date Range: Last 7 Days by @NANDINI-star in #12917
• Added handling for pwd protected cert files in AOAI CertificateCreden… by @Ne0-1 in #12995
• docs: add Qwen Code CLI tutorial by @colesmcintosh in #12915
• Azure api_version="preview" support + Bedrock cost tracking via Anthropic /v1/messages by @krrishdholakia in #13072
• Remove extraneous s in docs by @maxrabin in #13079
• Fix list team v2 security check by @tofarr in #13094
• [MCP gateway] add pre and during call hooks init by @jugaldb in #13067
• fix: improve MCP server URL validation to support internal/Kubernetes URLs by @colesmcintosh in #13099
• [Bug Fix] Gemini-CLI - The Gemini Custom API request has an incorrect authorization format by @ishaan-jaff in #13098
• set default value for mcp namespace tool name in spend table to prevent duplicate entry in table by @lvuna in #12894
• [Feat] Allow using query_params for setting API Key for generateContent routes by @ishaan-jaff in #13100
• [LLM translation] add openrouter grok4 by @jugaldb in #13018
• [Infra] Looses MCP python version restrictions by @ishaan-jaff in #13102
• fix: always use choice index=0 for Anthropic streaming responses by @Maximgitman in #12666
• BUGFIX: Jitter should be added not multiplied (#12877) by @pathikrit in #12901
• fix: helm migration job not running schema update by @huetterma in #12809
• [Bug]: Set user from token user_id for OpenMeter integration by @betterthanbreakfast in #13029
• Revert "[Bug]: Set user from token user_id for OpenMeter integration" by @krrishdholakia in #13107
• Fix fallback delete by @phosae in #12606
• Fix/gemini api key environment variable support by @sahusiddharth in #12507
• [MCP Gateway] Add protocol headers by @jugaldb in #13062
• Fix token counter to ignore unsupported keys like prefix (#11791) by @Amit-kr26 in #11954
• Custom Auth - bubble up custom exceptions by @krrishdholakia in #13093
• feat: Add dot notation support for all JWT fields by @colesmcintosh in #13013
• [Feat] MLFlow Logging - Allow adding tags for ML Flow logging requests by @ishaan-jaff in #13108
• [LLM translation] Add support for bedrock computer use by @jugaldb in #12948
• After selecting date range show loader on usage cost charts by @NANDINI-star in #13113
• Revert "[LLM translation] Add support for bedrock computer use" by @krrishdholakia in #13118
• fix(model_checks.py): handle custom values in wildcard model name (e.g. genai/test/*) by @krrishdholakia in #13116
• move to use_prisma_migrate by default + resolve team-only models on auth checks + UI - add sagemaker on UI by @krrishdholakia in #13117
• fix tool aws bedrock call index when the function only have optional arg by @jnhyperion in #13115
• [MCP Gateway] add health check endpoints for MCP by @jugaldb in #13106
• [MCP Protocol header] fix issue with clients protocol header by @jugaldb in #13112
• Added Voyage, Jinai, Deepinfra and VolcEngine providers on the UI by @NANDINI-star in #13131
• [Proxy UI] fix object permission for orgs by @jugaldb in #13142
• New Advanced Date Range Picker Component by @NANDINI-star in #13141
• [Feat] UI + Backend add a tab for use agent activity by @ishaan-jaff in #13146
• [LLM translation] Fix bedrock computer use by @jugaldb in #13143
• [MCP Guardrails] move pre and during hooks to ProxyLoggin by @jugaldb in #13109
• [Feat] v2 updates - tracking DAU, WAU, MAU for coding tool usage + show Daily Usage per User by @ishaan-jaff in #13147
• Litellm explore postgres db ci cd by @krrishdholakia in #13156
• [MCP Gateway] Litellm mcp client list fail by @jugaldb in #13114
• [MCP Gateway] fix migrations by @jugaldb in #13157
• build(config.yml): migrate build_and_test to ci/cd pg db by @krrishdholakia in #13166
• add framework name to UserAgent header in AWS Bedrock API call by @0x-fang in #13159
• fix: remove obsolete attribute version in docker compose by @kowyo in #13172
• [Feat] Background Health Checks - Allow disabling background health checks for a specific by @ishaan-jaff in #13186
• [Proxy Startup]fix db config through envs by @jugaldb in #13111
• fix: support negative indexes in cache_control_injection_points for Anthropic Claude (#10226) by @AnandKhinvasara in #13187
• [Bug Fix] Gemini-CLI Integration - ensure tool calling works as expected on generateContent by @ishaan-jaff in #13189
• [Bug Fix] Infra - ensure that stale Prisma clients disconnect DB connection by @ishaan-jaff in #13140
• [Feat] Allow redacting message / response content for specific logging integrations - DD LLM Observability by @ishaan-jaff in #13158
• Litellm fix fallbacks UI by @ishaan-jaff in #13191
• Fix langfuse test patch path causing CI failures by @colesmcintosh in #13192
• [Docs] Add details on when to use specific health endpoints by @ishaan-jaff in #13193
• Fix - using managed files w/ OTEL + UI - add model group alias on UI by @krrishdholakia in #13171
• Anthropic - working mid-stream fallbacks by @krrishdholakia in #13149
• Anthropic - mid stream fallbacks p2 (add token usage across both calls) by @krrishdholakia in #13170
• [UI QA] QA - Agent Activity Tab by @ishaan-jaff in #13203
• Fix/panw prisma airs post call hook by @jroberts2600 in #13185
• feat(helm): allow helm hooks for migrations job by @unique-jakub in #13174
• add openssl in apk install in runtime stage in dockerfile.non_root by @mdiloreto in #13168
• Allow to redifine LLM base api URL in the pass trough endpoints by @tyumentsev4 in #13134
• Fix API Key Being Logged by @aayush-malviya-acquia in #12978
• Index.md - cleanup docs by @krrishdholakia in #13215
• [LLM translation] Fix bedrock computer use #13143 by @jugaldb in #13150
• [QA] Viewing Agent Activity Headers on UI Usage Page by @ishaan-jaff in #13212
• [Separate Health App] Update Helm Deployment.yaml by @jugaldb in #13162
• [Proxy]fix key mgmt by @jugaldb in #13148
• [LLM] fix model reload on model update by @jugaldb in #13216
• [LLM] - suppress httpx logging by @jugaldb in #13217
• [MCP Gateway] Litellm mcp pre and during guardrails by @jugaldb in #13188

New Contributors

• @5731la made their first contribution in #12989
• @restato made their first contribution in #12980
• @strickvl made their first contribution in #12956
• @Ne0-1 made their first contribution in #12995
• @maxrabin made their first contribution in #13079
• @lvuna made their first contribution in #12894
• @Maximgitman made their first contribution in #12666
• @pathikrit made their first contribution in #12901
• @huetterma made their first contribution in #12809
• @betterthanbreakfast made their first contribution in #13029
• @phosae made their first contribution in #12606
• @sahusiddharth made their first contribution in #12507
• @Amit-kr26 made their first contribution in #11954
• @kowyo made their first contribution in #13172
• @AnandKhinvasara made their first contribution in #13187
• @unique-jakub made their first contribution in #13174
• @tyumentsev4 made their first contribution in #13134
• @aayush-malviya-acquia made their first contribution in #12978

Full Changelog: v1.74.9-stable...v1.74.14-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.74.14-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results