BerriAI/litellm v1.80.8.rc.1 on GitHub

What's Changed

merge main by @Sameerlite in #17408
Guardrail API - support tool call checks on OpenAI /chat/completions, OpenAI /responses, Anthropic /v1/messages by @krrishdholakia in #17459
docs: document responses and embedding api for github copilot by @codgician in #17456
fix(openai): add clear error message for empty LLM endpoint responses by @DevajMody in #17445
Fix: Skip empty text blocks in Anthropic system messages by @HaiyiMei in #17442
feat: Basic Weave OTEL integration by @andrewtruong in #17439
fix(ocr): add missing ocr and aocr to CallTypes enum by @Chesars in #17435
fix(groq): check supports_response_schema before using json_tool_call workaround by @Chesars in #17438
fix(pricing): add missing Moonshot turbo models and fix incorrect pricing by @Chesars in #17432
fix: make bedrock converse messages respect modify_params as expected by @danielmklein in #17427
Update docs to link agent hub by @krrishdholakia in #17462
[Feature] Add Auto Redirect to SSO to New Login Page by @yuneng-jiang in #17451
[Infra] UI - Change Credentials to use React Query by @yuneng-jiang in #17465
Map output_tokens_details of responses API to completion_tokens_details by @Sameerlite in #17458
Add support for bedrock qwen 2 imported model by @Sameerlite in #17461
Add support for file content download for bedrock batches by @Sameerlite in #17470
fix: test_default_api_base for ragfow by @Sameerlite in #17485
feat - new model amazon.titan-image-generator-v2:0 by @ishaan-jaff in #17488
Fix aim guardrail tests by @hxyannay in #17499
fix: resolve ruff lint errors by @AnasAbdelR in #17490
🆕 feat: support routing to only websearch supported deployments by @raghav-stripe in #17500
[Management endpoints] - Allow admin viewer to access global tag usage by @ishaan-jaff in #17501
[Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend by @ishaan-jaff in #17502
[Feat] UI - Agent Gateway - set allowed agents by key, team by @ishaan-jaff in #17511
[Docs] A2a - Permission management by @ishaan-jaff in #17515
[Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider by @ishaan-jaff in #17448
chore: add stack trace to mcp error message by @uc4w6c in #17269
(fix): empty response + vllm streaming by @dominicfeliton in #17516
fix(guardrails): mask all matching keywords in content filter by @DevajMody in #17521
Enable detailed debugging for reference by @krrishdholakia in #17508
fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage by @kristianmitk in #17504
fix(gemini): handle partial JSON chunks after first valid chunk by @Chesars in #17496
fix(unified_guardrail.py): support during_call event type for unified guardrails by @krrishdholakia in #17514
Guardrails API - new structured_messages param by @krrishdholakia in #17518
[Feat] VertexAI - Add BGE Embeddings support by @Sameerlite in #17362
[Fix] Select in Edit Membership Modal by @yuneng-jiang in #17524
fix: code quality issues from ruff linter by @AlexsanderHamir in #17536
Fix: Presidio guardrail test TypeError and license base64 decoding error by @AlexsanderHamir in #17538
[fix] parse budget:thinking blocks for opus 4.5 by @colinlin-stripe in #17534
Fix vector store configuration synchronization failure by @Sameerlite in #17525
Add support for cursor BYOK with its own configuration by @Sameerlite in #17519
Add background health checks to db by @Sameerlite in #17528
Fix: Lack of None value checks & update publicai_chat_transformation tests by @AlexsanderHamir in #17539
Remove hardcoded api key from test file by @krrishdholakia in #17540
Make sure that media resolution is only for gemini 3 model by @Sameerlite in #17137
[Bug fix] WatsonX audio transcriptions, don't force content type in request headers by @ishaan-jaff in #17546
[Infra] Bump LiteLLM Enterprise Version by @yuneng-jiang in #17549
Fix budget update to allow null max_budget by @ishaan-jaff in #17545
fix failing vertex tests by @Sameerlite in #17542
[Feature] Show all callbacks on UI by @yuneng-jiang in #16335
fix(github_copilot): preserve encrypted_content in reasoning items for multi-turn conversations by @rgshr in #17130
feat: Add gpt-5.1-codex-max model pricing and configuration by @Chesars in #17541
Fix: apply_guardrail method and improve test isolation by @AlexsanderHamir in #17555
Fix: MLflow streaming spans for Anthropic passthrough by @uc4w6c in #17288
[Fixes] Dynamic Rate Limiter - Dynamic rate limiting token count increases/decreases by 1 instead of actual count + Redis TTL by @ishaan-jaff in #17558
[Bug fix] Secret Managers Integration - Make email and secret manager operations independent in key management hooks by @ishaan-jaff in #17551
feat: add Mistral Large 3 model support by @Chesars in #17547
fix(team): use organization.members instead of deprecated organizatio… by @DevajMody in #17557
[Feature] Customer (end user) Usage by @yuneng-jiang in #17498
[Feat] Allow using dynamic rate limit/priority reservation on teams by @ishaan-jaff in #17061
build(deps): bump jws from 3.2.2 to 3.2.3 in /ui/litellm-dashboard by @dependabot[bot] in #17494
fix(presidio): handle empty content and error dict responses in guardrails by @dominicfallows in #17489
fix(bedrock): anthropics beta headers for qwen by @irfansofyana in #17467
fix(gemini): fix reasoning_content showing duplicate content in streaming responses by @Chesars in #17266
Fixes #13652 - auth not working with ollama.com by @GusBricker in #17191
fix(docs): remove source .env by @utsumi-fj in #17466
fix(streaming): add length validation for empty tool_calls in delta by @DevajMody in #17523
feat(oci): Enable Oracle Cloud Infrastructure configuration via UI by @jatorre in #17159
Fix: Gemini image_tokens treated as text tokens in cost calculation by @Chesars in #17554
fix(responses): Add image generation support for Responses API by @Chesars in #16586
fix: Handle global location for Vertex AI image generation endpoint by @OlivverX in #17255
Feat/mcp preserve tool metadata calltoolresult by @uc4w6c in #17561
[Feature] Customer Usage UI by @yuneng-jiang in #17506
[Fix] Mock server_root_path for v2/login test by @yuneng-jiang in #17563
[Fix] CI/CD - Adding end user and org to service types by @yuneng-jiang in #17564
build(deps): bump mdast-util-to-hast from 13.2.0 to 13.2.1 in /ui/litellm-dashboard by @dependabot[bot] in #17444
[Feature] Support Images in Compare UI by @yuneng-jiang in #17562
fix: mcp test by @uc4w6c in #17570
[Fix] Flaky UI Test by @yuneng-jiang in #17569
[Fix] Change useAuthorized Hook to redirect to new Login Page by @yuneng-jiang in #17553
Fix: remove merge conflict markdown by @AlexsanderHamir in #17586
Fix Langfuse logger test mock setup by @AlexsanderHamir in #17588
fix: preserve usage from JSON response for OpenAI provider in Bedrock by @AlexsanderHamir in #17589
Revert "Fix Langfuse logger test mock setup (#17588)" by @AlexsanderHamir in #17590
Fix: Langfuse logger test mock setup by @AlexsanderHamir in #17591
Fix flaky test: test_logging_non_streaming_request by @AlexsanderHamir in #17592
Fix: Ensure guardrail metadata is preserved in request_data by @AlexsanderHamir in #17593
Fix: Resolve flakiness in three integration tests by @AlexsanderHamir in #17594
fix: add retry logic for flaky Azure image generation health check test by @AlexsanderHamir in #17595
Add retry logic to apk package installation in Dockerfile.non_root by @AlexsanderHamir in #17596
fix: make test_watsonx_gpt_oss_prompt_transformation deterministic by @AlexsanderHamir in #17597
[Feature] UI - Add Info Banner for Customer Usage by @yuneng-jiang in #17598
fix: make test_log_langfuse_v2_handles_null_usage_values more robust by @AlexsanderHamir in #17599
[Infra] Building UI for release by @yuneng-jiang in #17600
Fix security vulnerability: update mdast-util-to-hast to 13.2.1 by @AlexsanderHamir in #17601
fix: mark test_caching_with_reasoning_content as flaky and handle InternalServerError by @AlexsanderHamir in #17603
docs: add gpt-5.1-codex-max to OpenAI provider documentation by @Chesars in #17602
feat: Add xhigh reasoning effort for gpt-5.1-codex-max by @withsmilo in #17585
1.80.8 RC docs by @ishaan-jaff in #17605
feat: add script to create branches with litellm_ prefix by @AlexsanderHamir in #17606
[Fix] UI - MCP Test Tool Fix by @yuneng-jiang in #17609
[Infra] Building UI To Include Fix by @yuneng-jiang in #17610
Fix: test_guardrails_with_api_key_controls by @AlexsanderHamir in #17612
refactor: extract helper methods from pre_call_hook to fix PLR0915 by @AlexsanderHamir in #17613
[Docs] Customer Usage by @yuneng-jiang in #17615
docs: add Microsoft GraphRAG to projects using LiteLLM by @Chesars in #17616
Helm Chart: Add possibility to override command, args and add deployment labels by @LukasdeBoer in #17535
[Docs] Update sidebar for customer usage by @yuneng-jiang in #17617
[New Model] Add Amazon Nova as first party provider for chat completions by @anilkumarkodali in #17351
[Docs] Fixing path to image by @yuneng-jiang in #17618
Add beta header in vertex request for claude websearch request by @Sameerlite in #16269
fix(vertex_llm_base.py): Add missing scopes to identity pool credentials by @krrishdholakia in #17560
[Feature] Loading states for Edit Membership modal by @yuneng-jiang in #17621

New Contributors

@DevajMody made their first contribution in #17445
@andrewtruong made their first contribution in #17439
@AnasAbdelR made their first contribution in #17490
@dominicfeliton made their first contribution in #17516
@kristianmitk made their first contribution in #17504
@rgshr made their first contribution in #17130
@dominicfallows made their first contribution in #17489
@irfansofyana made their first contribution in #17467
@GusBricker made their first contribution in #17191
@OlivverX made their first contribution in #17255
@withsmilo made their first contribution in #17585
@LukasdeBoer made their first contribution in #17535
@anilkumarkodali made their first contribution in #17351

Full Changelog: v1.80.7.dev.3...v1.80.8.rc.1