BerriAI/litellm v1.80.16.dev6 on GitHub

What's Changed

implement failopen option default to True on grayswan guardrail by @Reapor-Yurnero in #18266
fix(otel): update semantic conventions to 1.38 (gen_ai attributes) by @Harshit28j in #18793
merge main by @Sameerlite in #18962
fix: case-insensitive model cost map lookup by @Chesars in #18208
Add pricing of azure_ai/claude-opus-4-5 by @Sameerlite in #19003
Fix image tokens spend logging for /images/generations by @Dima-Mediator in #19009
Staging 12/19/2025 - implement failopen option default to True on grayswan guardrail (#18266) by @krrishdholakia in #18976
staging 01/09/2025 by @krrishdholakia in #18873
Fix: Header forwarding in bedrock passthrough by @Sameerlite in #19007
Fix: Model matching priority in configuration by @Sameerlite in #19012
[Feat] Add support for 0 cost models by @Sameerlite in #19027
fix(langsmith.py): hoist thread grouping metadata (session_id, thread… by @Harshit28j in #18982
fix: properly handle custom guardrails parameters by @Harshit28j in #18978
fix(ui): use non-streaming method for endpoint v1/a2a/message/send in… by @houdataali in #19025
Fix: update novita models prices by @ji-chuan-che in #19005
Normalize OpenAI SDK BaseModel choices/messages to avoid Pydantic serializer warnings by @hummat in #18972
Fix: Prompt deletion fails with Prisma FieldNotFoundError by @berkeyalciin in #18966
chore: allow passing scope id for watsonx inferencing by @MateuszOssGit in #18959
Fix Swagger UI path execute error with server_root_path in OpenAPI schema by @xfan001 in #18947
fix: correct context window sizes for GPT-5 model variants by @Chesars in #18928
fix(ollama): set finish_reason to tool_calls and remove broken capability check by @rsp2k in #18924
fix(model_prices): sync DeepSeek chat/reasoner to V3.2 pricing by @nulone in #18884
fix(exception_mapping): handle exceptions without response parameter by @debnil-mercor in #18919
fix: num_retries in litellm_params (#18968) as per config by @Harshit28j in #18975
fix(pricing): correct cache_read pricing for gemini-2.5-pro models by @Chesars in #18157
feat(gemini): add opt-in support for responseJsonSchema by @Chesars in #18147
[Perf] Remove bottleneck causing high CPU usage & overhead under heavy load by @AlexsanderHamir in #19049
[Refactor] UI - User and Team Table Filters to Reusable Component by @yuneng-jiang in #19010
[Feature] UI - Simplify Key Generate Permission Error by @yuneng-jiang in #18997
[Fix] UI - Usage: Team ID and Team Name in Export Report by @yuneng-jiang in #19047
[bug fix] do not fallback to token counter if disable_token_counter is enabled by @raghav-stripe in #19041
[Feature] UI - Usage: Allow Top Virtual Keys and Models to Show More Entries by @yuneng-jiang in #19050
[Refactor] Add CI enforcement for O(1) operations in _get_model_cost_key to prevent performance regressions by @AlexsanderHamir in #19052
[fix] mcp rest auth checks by @uc4w6c in #19051
[Feat] New Model - Azure Model Router on LiteLLM AI Gateway by @ishaan-jaff in #19054
[Docs] Add Litellm architecture md doc by @ishaan-jaff in #19057
[fix] generate two telemetry in responses by @uc4w6c in #18938
feat: Add Cerebras zai-glm-4.7 model support and deprecate zai-glm-4.6 by @Chesars in #18864
[Fix] UI - Usage: Model Activity Chart Y Axis by @yuneng-jiang in #19055
[Fix] Feature Flag for Fix SCIM GET /Users error and enforce SCIM 2.0 compliance by @yuneng-jiang in #18878
Fix SCIM GET /Users error and enforce SCIM 2.0 compliance by @hakhundov in #17420
[docs] auth message UI by @uc4w6c in #19063
[fix] anthropic during call error by @uc4w6c in #19060
fix: enable JSON logging via configuration and add regression test by @Jetemple in #19037
fix(guardrails): use clean error messages for blocked requests by @eagle-p in #19023
fix: add handling for user-disabled mid-stream fallbacks by @rohanwinsor in #19078
merge main by @Sameerlite in #19079
Revert "feat(gemini): add opt-in support for responseJsonSchema" by @Sameerlite in #19080
feat: contextual gap checks, word-form digits by @uc4w6c in #18301
Fix: Responses content can't be none by @Sameerlite in #19064
Fix: anthropic token counter with thinking by @Sameerlite in #19067
Fix: [Bug]: Gemini Image Generation Returns Incorrect prompt_tokens_d… by @Sameerlite in #19070
Add retry policy support to responses API by @Sameerlite in #19074
fix(models): correct max_input_tokens for GPT-5 models by @Chesars in #19056
Staging 01/13/2026 by @krrishdholakia in #19042
fix(dynamic_rate_limiter_v3): fix TPM 25% limiting by ensuring priori… by @Harshit28j in #19092
feat(proxy): cleanup spend logs cron verification, fix, and docs by @Harshit28j in #19085
fix(vertex_ai): improve passthrough endpoint url parsing and construction (#17402) by @krisxia0506 in #17526
fix(gemini): dereference $defs/$ref in tool response content by @debnil-mercor in #19062
docs: Add troubleshooting guide by @AlexsanderHamir in #19096
fix: preserve llm_provider-* headers in error responses by @pgolm in #19020
fix: keep type field in Gemini schema when properties is empty by @theonlypal in #18979
docs: Update troubleshooting guide by @AlexsanderHamir in #19097
update troubleshoot docs by @AlexsanderHamir in #19099
Fix Azure Grok prices by @emerzon in #19102
fix(vertex): add type object to tool schemas missing type field by @theonlypal in #19103
Add support for OpenAI's gpt-5.2-codex by @PeterDaveHello in #19101
fix: model hub client side exception by @dominicfeliton in #19045
[Fix] Claude Code + Bedrock Converse Usage - ensure budget tokens are passed to converse api correctly by @ishaan-jaff in #19107
[Perf] Remove premature model.dump call on the hot path by @AlexsanderHamir in #19109
[Feature] UI - Model: Anthropic Models QOL by @yuneng-jiang in #19058
[Fix] /team/daily/activity Show Internal Users Their Spend Only by @yuneng-jiang in #19053
[Fix] Claude Code (/messages) - Litellm fix claude code Bedrock Invoke usage, request signing by @ishaan-jaff in #19111
[Fix] Containers API - Container API routes return 401 for non-admin users - routes missing from openai_routes by @ishaan-jaff in #19115
docs: Add structured issue reporting guides for CPU and memory issues by @AlexsanderHamir in #19117
[Feature] UI - Edit Key: Add search to key edit team dropdown by @yuneng-jiang in #19119
[doc] add MCP troubleshooting guide by @uc4w6c in #19122
[Fix] Containers API - Allow routing to regional endpoints by @ishaan-jaff in #19118
[Feat] Claude Code - Add support for Prompt Caching with Bedrock Converse by @ishaan-jaff in #19123
[Fix] Azure Storage Circular Reference Error by @yuneng-jiang in #19120
[Fix] Add support for caching for responses API by @Sameerlite in #19068
bump litellm version to 0.1.28 by @Sameerlite in #19127
[fix] mcp chat completions by @uc4w6c in #19129
Add openrouter support for image/generation endpoints by @Sameerlite in #19059
docs(logging.md): add guide for mounting custom callbacks in Helm/K8s by @Harshit28j in #19136
fix: mount config.yaml as single file in Helm chart by @Harshit28j in #19146
Stabilise mock tests by @Sameerlite in #19141
Fix model name from query param in realtime request by @Sameerlite in #19135
Added ability to customize logfire base url through env var by @vikigenius in #19148
Litellm dev 01 15 2026 p1 by @krrishdholakia in #19153
chore: clean up orphaned files from repository root by @Chesars in #19150
Update the version of boto3 to 1.40.15 and aioboto3 to 15.5.0 by @burnerburnerburnerman in #19090
Add max_tokens settings for volcengine models (deepseek-v3-2, glm-4-7, kimi-k2-thinking) by @yfge in #19076
Make keepalive_timeout parameter work for Gunicorn by @krisxia0506 in #19087
Update prisma_migration.py by @danielnyari-seon in #19083
fix: model-level guardrails not taking effect (#18363) by @Harshit28j in #18895
fix: models loadbalancing billing issue by filter by @Harshit28j in #18891
fix: video status/content credential injection for wildcard models by @choigawoon in #18854
[Feature] UI - Community Engagement Buttons by @yuneng-jiang in #19114
[Feature] UI - Allow Preventing Team Admins From Deleting Members From Teams by @yuneng-jiang in #19128
[Fix] /user/new Privilege Escalation by @yuneng-jiang in #19116
[Feature] User Metrics for Promethus by @yuneng-jiang in #18785
Fix Azure embeddings JSON parsing to prevent connection leaks and ensure proper router cooldown by @AlexsanderHamir in #19167
[Feat] Add support for Tool Search on /messages API - Azure, Bedrock, Anthropic API by @ishaan-jaff in #19165
Revert "[Feat] Add support for 0 cost models" by @uc4w6c in #19168
[Feature] UI - Reusable Model Select by @yuneng-jiang in #19164
Litellm fix ci check code and doc quality by @uc4w6c in #19170
[Feat] Claude Code - Add End-user tracking with Claude Code by @ishaan-jaff in #19171
Revert "fix(gemini): dereference $defs/$ref in tool response content" by @uc4w6c in #19173
[Docs Guide] Litellm claude code end user tracking by @ishaan-jaff in #19176
fix gcp glm-4.7 pricing by @emerzon in #19172
Improve documentation for routing LLM calls via SAP Gen AI Hub by @guilherme-segantini in #19166
fix(bedrock): strip throughput tier suffixes from model names by @jgreek in #19147
Revert "[Fix] /team/daily/activity Show Internal Users Their Spend Only" by @uc4w6c in #19175
Revert "[Perf] Remove premature model.dump call on the hot path" by @uc4w6c in #19177
[tests] skip Azure SDK init check for acreate_skill by @uc4w6c in #19178
[test] handle wildcard routes in route validation test by @uc4w6c in #19182
Fix : test_stream_chunk_builder_litellm_mixed_calls by @Sameerlite in #19179
[chore] document temporary grype ignore for CVE-2026-22184 by @uc4w6c in #19181
[chore] add jaraco liccheck by @uc4w6c in #19188
bump: version 1.80.16 → 1.80.17 by @uc4w6c in #19189
[chore]: add ALLOWED_CVES by @uc4w6c in #19200
merge main by @Sameerlite in #19203
Revert "fix: models loadbalancing billing issue by filter" by @Sameerlite in #19205
merge main by @Sameerlite in #19206
Litellm staging 01 14 2026 by @krrishdholakia in #19095
merge main in bedrock passthrough by @Sameerlite in #19210
merge main in fallback PR by @Sameerlite in #19211
Add medium value support for detail param for gemini by @Sameerlite in #19187
merge main in timeout PR by @Sameerlite in #19212
merge main in sanitisation by @Sameerlite in #19213
merge main in malformed tool call PR by @Sameerlite in #19214
merge main in stability issue by @Sameerlite in #19215
Fix: vertex ai doesn't support structured output by @Sameerlite in #19201
[Feat] Add fallback endpoints support by @Sameerlite in #19185
Fix: [Bug]: stream_timeout：The function of this parameter has been changed by @Sameerlite in #19191
Add user auth in standard logging object for bedrock passthrough by @Sameerlite in #19140
Add sanititzation for anthropic messages by @Sameerlite in #19196
Litellm staging 01 15 2026 by @krrishdholakia in #19159
Fix: malformed tool call transformation in bedrock by @Sameerlite in #19198
[Fix] Bedrock stability model usage issues by @Sameerlite in #19199
Add azure/gpt-5.2-codex by @Sameerlite in #19219

New Contributors

@houdataali made their first contribution in #19025
@hummat made their first contribution in #18972
@berkeyalciin made their first contribution in #18966
@MateuszOssGit made their first contribution in #18959
@xfan001 made their first contribution in #18947
@nulone made their first contribution in #18884
@debnil-mercor made their first contribution in #18919
@hakhundov made their first contribution in #17420
@rohanwinsor made their first contribution in #19078
@pgolm made their first contribution in #19020
@vikigenius made their first contribution in #19148
@burnerburnerburnerman made their first contribution in #19090
@yfge made their first contribution in #19076
@danielnyari-seon made their first contribution in #19083
@guilherme-segantini made their first contribution in #19166
@jgreek made their first contribution in #19147

Full Changelog: v1.80.16-nightly...v1.80.16.dev6