BerriAI/litellm v1.74.1-nightly

on GitHub

What's Changed

• Add mcp server segregation comma separated support by @jugaldb in #12326
• Fix: Preserve Live Tail State on Log Pages by @NANDINI-star in #12335
• [Feat] JWT - Sync user roles and team memberships when JWT Auth is used by @ishaan-jaff in #11994
• fix watsonx datetime conversion issue py3.10 by @isaken in #12339
• Patch 1 by @isaken in #12338
• UI - Add Azure Content Safety Guardrails (improved UX) by @krrishdholakia in #12330
• UI - Azure Content Guardrails by @krrishdholakia in #12341
• feat(vertex_ai/): add new deepseek-ai api service by @krrishdholakia in #12312
• v1.74.0.rc docs by @ishaan-jaff in #12344
• [Docs] vertex deepseek by @ishaan-jaff in #12345
• docs - 1.74.0.rc by @ishaan-jaff in #12347
• [UI QA] 1.74.0.rc by @ishaan-jaff in #12348
• fix: add proper type annotations for embedding() function by @colesmcintosh in #12262
• Remove stream options from streaming + fix guardrail start time on log duration by @krrishdholakia in #12346
• Add all guardrails to the UI by @krrishdholakia in #12349
• New /key/service-account/generate API Endpoint + Team member permissions for creating service account keys by @krrishdholakia in #12350
• Sticky session for Test Key page by @NANDINI-star in #12365
• Truncate long labels and improve tooltip in Top API Keys chart by @NANDINI-star in #12371
• [Bug Fix] s3 config.yaml file - ensure yaml safe load is used by @ishaan-jaff in #12373
• [Feat] Bump langfuse python SDK version and LANGFUSE_TRACING_ENVIRONMENT by @ishaan-jaff in #12376
• [Security] Bump mcp version on docker img by @ishaan-jaff in #12362
• fix: make TextCompletionStreamWrapper conversion retain reasoning_content by @aholmberg in #12377
• Bump mcp from 1.9.3 to 1.10.0 by @dependabot in #12388
• [Feat] Add MCP Cost Tracking by @ishaan-jaff in #12385
• feat: add image support for Responses API when falling back on Chat Completions by @ryan-castner in #12204
• Add 'thinking blocks' to stream chunk builder + remove experimental 'by_tag' metrics on prometheus (fix cardinality issue) by @krrishdholakia in #12395
• Add audit logs on model update by @krrishdholakia in #12396
• Improve Chart Readability for Tags by @NANDINI-star in #12378
• Fix API base url for Github Copilot provider by @kanaka in #12418
• fix(proxy/mcp): Error handling MCP request: Task group is not initialized by @juancarlosm in #12411
• style: update sambanova logos by @jhpiedrahitao in #12431
• [Bug fix] MCP MCP_TOOL_PREFIX_SEPARATOR to work with claude code by @jugaldb in #12430
• Prevent navigation reset after team member operations by @NANDINI-star in #12424
• Fix guardrails_ai.md documentation page by @DmitriyAlergant in #12356
• [Bug fix] Multiple API Keys Created on Startup When max_budget is Enabled by @ishaan-jaff in #12436
• [Feat] Add XInference Image Generation API Provider by @ishaan-jaff in #12439
• [Feat] Bedrock Guardrails - Raise Bedrock output text on 'BLOCKED' actions from guardrail by @ishaan-jaff in #12435
• MCP - usage tracking by @krrishdholakia in #12397
• fix(utils.py): rollback faulty security check on files by @krrishdholakia in #12441
• Fix: Properly close aiohttp client sessions to prevent resource leaks by @colesmcintosh in #12251
• Remove temporary test files by @colesmcintosh in #12442
• (Router) don't add invalid deployment to router pattern match by @krrishdholakia in #12459
• [Feat] MCP Gateway - Allow customizing what client side header to use by @ishaan-jaff in #12460
• [Bug Fix] Ensure supported bedrock/converse/ params = bedrock/ params by @ishaan-jaff in #12466
• Litellm mcp internal users by @jugaldb in #12458
• [Feat] SSO - Allow users to run a custom sso login handler by @ishaan-jaff in #12465
• [Bug Fix] DataDogLLMObsLogger push total_cost by @ishaan-jaff in #12467
• [MCP Gateway] - Allow using custom post call MCP hook for cost tracking by @ishaan-jaff in #12469
• DB Spend Update Writer: fix query + Allow anthropic-beta header when forward_client_headers_to_llm_api is true by @krrishdholakia in #12462
• OTEL - OTEL_RESOURCE_ATTRIBUTES support + Model Hub - new model hub table view, new /public/model_hub endpoint, fix duplicates in /model_group/info by @krrishdholakia in #12468
• Resolve model group alias on Auth + /v1/messages Fallback support by @krrishdholakia in #12440

New Contributors

• @isaken made their first contribution in #12339
• @kanaka made their first contribution in #12418
• @juancarlosm made their first contribution in #12411
• @DmitriyAlergant made their first contribution in #12356

Full Changelog: v1.74.0-nightly...v1.74.1-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.74.1-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results