BerriAI/litellm v1.98.0-dev.2 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.98.0-dev.2

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.98.0-dev.2/cosign.pub \
  ghcr.io/berriai/litellm:v1.98.0-dev.2

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

fix(batches): attribute Vertex passthrough batch cost to key/team/tags by @yucheng-berri in #34456
docs: rewrite the CLAUDE.md comment rule with explicit exceptions by @devin-ai-integration[bot] in #36301
fix(proxy): scope file list pagination cursors to the caller by @devin-ai-integration[bot] in #36093
fix(proxy): skip prisma-dependent hooks when no database is attached by @mateo-berri in #36273
fix(proxy): report has_more false on caller-scoped file list pages by @mateo-berri in #36326
fix(proxy): restore management_v1 query-param validation under fastapi>=0.140.7 by @HuanQian571 in #35773
fix(proxy): stop /{provider}/v1/files from capturing /openai_passthrough by @devin-ai-integration[bot] in #36092
chore(typing): remove 914 basedpyright Any errors across 16 hotspot files by @mateo-berri in #36386
fix(router): keep batch fallbacks inside the model group that owns the file by @mateo-berri in #36181
feat(ptu): configure provisioned-throughput flat cost on a model deployment by @yucheng-berri in #35341
docs: clarify the CLAUDE.md comment exceptions are any-of by @devin-ai-integration[bot] in #36421
docs: replace the Changes PR template section with Caveats by @devin-ai-integration[bot] in #36423
fix(bedrock): enable native structured output for GLM 5 and DeepSeek V3.2 by @alexshtf in #35669
feat(ptu): daily rollup writes per-model PTU flat cost by active hour by @yucheng-berri in #35343
feat(logging): add opt-in session_id and trace_id correlation to JSON log records via contextvars by @deepanshululla in #34418
feat(ptu): surface PTU flat cost on the daily activity read path by @yucheng-berri in #35391
feat(router): add per-deployment allowed_fails_policy and cooldown_time override support by @deepanshululla in #34416
feat(ptu): add PTU inputs to the model form and flat cost to the Usage page by @yucheng-berri in #35393
fix(cost): price dict-shaped image input token details at the image rate by @vairodp in #33490
fix(model_prices): refresh deprecation dates, correct xAI pricing and add missing provider models by @devin-ai-integration[bot] in #36403
feat(ptu): gate PTU flat-cost attribution behind an opt-in env var by @yucheng-berri in #36138
ci: cache Prisma CLI and engine binaries, split test timeout from setup by @mateo-berri in #36417
feat(rate limiting): configurable estimated output tokens per key, team and model by @yassin-berriai in #36143
fix(ui): hide admin-only Logs tabs from roles that cannot call their endpoints by @yuneng-berri in #36333
test(proxy): guard management_v1 against fastapi names removed in supported releases by @yuneng-berri in #36336
fix(ui): gate policy and prompt lookups on an admin capability by @yuneng-berri in #36335
build(deps): bump pypdf to 6.15.0 to clear osv-scan by @devin-ai-integration[bot] in #36350
fix(proxy): isolate guardrail load failures per row by @yucheng-berri in #36432
fix(ui): gate organization and agent usage views behind capabilities by @yuneng-berri in #36334
fix(reset_budget_job): atomic budget cascade with chunked reset scans by @ryan-crabbe-berri in #36287
feat(proxy): add GET /v1/indexes to list vector store indexes by @ryan-crabbe-berri in #36289
feat(ui): show vector store indexes on the Vector Stores page by @ryan-crabbe-berri in #36306
fix(proxy): treat SAML as configured in UI SSO detection by @fancybear-dev in #36196
fix(bedrock): reject Anthropic server-side web_search tool with actionable error by @ryan-crabbe-berri in #36473
fix(ui): open the classifier prompt editor above the edit auto-router form by @tin-berri in #36438
fix(arize): trace MCP tool calls instead of crashing on CallToolResult by @yucheng-berri in #36453
refactor(ui): make illegal DataTable prop combinations unrepresentable by @yuneng-berri in #36470
fix(ui): scope Virtual Keys and Logs team lists to the caller by @yuneng-berri in #36472
fix(ui): gate the Old Usage page behind a proxy-admin capability by @yuneng-berri in #36469
docs(terraform): describe the provider release as automatic by @yuneng-berri in #36467
feat(proxy): add per-deployment keepalive_seconds SSE heartbeat to prevent load-balancer timeout on long streams by @deepanshululla in #34423
fix(router): cool down failed fallback deployments and correct cooldown TTL after Redis backfill by @deepanshululla in #35104
perf(spend): write each daily spend batch in one upsert statement by @yassin-berriai in #36448
fix(ui): gate four sidebar pages on the roles their endpoints allow by @yuneng-berri in #36475
fix(ui): restore the Logs Deleted Teams tab for organization admins by @yuneng-berri in #36478
fix(websearch): stop leaking interception control fields to providers by @yassin-berriai in #36480
test(e2e): cover the Anthropic web_search server tool on Bedrock by @yassin-berriai in #36443
fix(router): warn when a deployment's credentials contradict its provider by @yassin-berriai in #36486
fix: net prompt-caching savings against the cache-write premium by @tin-berri in #36452
feat(ui): deployment affinity toggle for the auto-router by @tin-berri in #36302
fix(bedrock): use deployment credentials for AWS requests by @daleselaji-dev in #36160
fix(anthropic): preserve midturn system corrections by @eugene-yao-zocdoc in #34290
fix(email): stop duplicate legacy invitation email and fix its onboarding link by @mubashir1osmani in #36455
feat(ui): show models under each tier in routing benchmark chart by @tin-berri in #36291
fix(proxy): inject streaming usage cost on openai passthrough streams by @mateo-berri in #36503
docs: require a user flow and live-proxy proof in bug reports by @devin-ai-integration[bot] in #36498
fix(proxy): add config_updated_at audit timestamp for virtual keys by @ryan-crabbe-berri in #36488
docs: require a user flow and a stuck-at proof in feature requests by @devin-ai-integration[bot] in #36500
feat(router): add required-AND (&) tag prefix and allow_fail_open flag by @deepanshululla in #36193
feat(proxy): per-key prompt caching toggle via enable_prompt_caching by @ryan-crabbe-berri in #36466
fix(bedrock): send tool-search beta header for Haiku 4.5 on Invoke /v1/messages by @mateo-berri in #36502
fix(bedrock): preserve adaptive thinking effort through the /v1/messages bridge by @mateo-berri in #36507
ci: retry transient network fetch failures in lint workflow by @mateo-berri in #36563
fix(ui): stub useIsOrgAdmin in UsageTab tests so useCan needs no QueryClient by @ryan-crabbe-berri in #36565
fix(alerting): dedupe scheduled Slack spend reports across pods by @ryan-crabbe-berri in #36489
chore(typing): clear 1.6k basedpyright Any errors across 56 files by @mateo-berri in #36543
fix(bedrock): add text block to converse user messages carrying documents by @mateo-berri in #36499
fix(deps): ship boto3 with the base SDK so bedrock works out of the box by @mubashir1osmani in #36568
fix(model_prices): add provider-announced deprecation dates for Bedrock, Mistral, Cohere and Gemini models by @devin-ai-integration[bot] in #36538
chore: bump litellm-enterprise 0.1.54 -> 0.1.55, litellm-proxy-extras 0.4.84 -> 0.4.85, litellm 1.97.0 -> 1.98.0 by @yuneng-berri in #36577
fix(bedrock_guardrails): skip ApplyGuardrail when there is no content to scan by @yucheng-berri in #36441
fix(e2e): assert on the gen-AI span that served the stream, not the span count by @yassin-berriai in #36582
test(e2e): harden vendor API coverage by @mubashir1osmani in #34557
test(e2e): add reproducers for passthrough and model budget gaps by @mubashir1osmani in #34657
test(e2e): cover google-native generateContent framing and prometheus queue time by @mubashir1osmani in #34650
chore(ci): promote internal staging to main by @tin-berri in #36560
feat(router): make routing groups callable as virtual models and list them in /v1/models by @tin-berri in #36519
fix(xai): bill web_search from server_side_tool_usage_details by @geraint0923 in #30817
fix(responses): init completed_response on bridge streaming iterator (#35411) by @devin-ai-integration[bot] in #35413
fix(batches): attribute Anthropic passthrough batch cost to the creating key, team and tags by @yucheng-berri in #36468
feat(dashscope): add latest Model Studio models to the cost map by @devin-ai-integration[bot] in #36496
fix(proxy): track streamed passthrough Responses cost by @william-xue in #36529
fix(model_prices): advertise native structured output on every Bedrock DeepSeek V3.2 and GLM 5 id by @yuneng-berri in #36597
test(bedrock): repoint live Claude tests off the retired Claude 3 Sonnet by @yuneng-berri in #36600
fix(anthropic): preserve speed=fast in usage for /v1/messages and pass-through by @devin-ai-integration[bot] in #36447
fix(proxy): forward resolved provider and deployment pricing in /cost/estimate by @devin-ai-integration[bot] in #35880
feat(proxy): global SSE keepalive ping interval for OpenAI-shaped streaming routes by @devin-ai-integration[bot] in #36154
fix(responses): preserve Codex namespace tool calls by @dcadenas in #32536
fix(nvidia_nim): preserve image passages and stop sending top_k to /v1/ranking by @atomic in #34177
fix: refactor HTTP handler initialization with client support by @Praveen11558 in #30952
feat(lint): gate writable TypedDict fields with LIT012 by @mateo-berri in #36590
perf(proxy): stagger scheduled background jobs across jobs and pods by @yassin-berriai in #36589
test: remove four mirror test files that exercise none of their module by @yuneng-berri in #34635
fix(router): stop re-applying router-selecting request tags to the routed tier's deployments by @mateo-berri in #36628
test: remove tests that never execute by @yuneng-berri in #36681
fix(ui): align spend and budget columns by @daniel-meismer-zocdoc in #35176
test: rename tests that a later definition shadowed by @yuneng-berri in #36685
fix(passthrough): carry the budget reservation into request metadata by @yassin-berriai in #36592
fix(mcp): bound MCP client requests with a session read timeout by @yassin-berriai in #36675
fix(proxy): log requests rejected for an unparsable body in spend logs by @yassin-berriai in #36673
refactor(ui): migrate cost-optimization to shadcn by @yuneng-berri in #36629
refactor(ui): migrate cost-tracking to shadcn by @yuneng-berri in #36631
refactor(ui): migrate admin-panel to shadcn by @yuneng-berri in #36635
refactor(ui): migrate users dashboard to shadcn by @yuneng-berri in #36642
refactor(ui): migrate prompts to shadcn by @yuneng-berri in #36643
refactor(ui): migrate team settings to shadcn by @yuneng-berri in #36641
refactor(ui): migrate models-and-endpoints to shadcn by @yuneng-berri in #36648
refactor(ui): migrate policy impact popover to shadcn by @yuneng-berri in #36653
fix(proxy): expand config-defined model access groups when resolving team models for /v2/model/info by @ryan-crabbe-berri in #34211
fix(batches): strip NUL bytes from passthrough batch tags before the managed object write by @yucheng-berri in #36688
test(e2e-ui): verify UI mutations against the API instead of trusting the toast by @yuneng-berri in #36632
fix(proxy): serialize model reconciles so concurrent model writes stop evicting each other by @yuneng-berri in #36687
chore(e2e): port the compat-matrix cron publisher to tests/e2e/claude_code by @mateo-berri in #36465
fix(router): never price a strategy-router alias by @tin-berri in #36691
feat(model_prices): add NVIDIA Nemotron 3.5 Lightning on OpenRouter and DeepInfra by @devin-ai-integration[bot] in #36696
feat(terraform/aws): make VPC, Aurora, and Redis optional by @yassin-berriai in #36676
feat(ui): warn in the Admin UI when no Redis is configured by @devin-ai-integration[bot] in #36495
fix(ui): show and edit key-level router settings on a virtual key by @yassin-berriai in #36674
fix(router): forward auto-router alias params from the marker entry, not the first same-name deployment by @mateo-berri in #36626
fix(bedrock_mantle): 1M context window and long-context pricing for GPT-5.6 Sol/Terra/Luna by @devin-ai-integration[bot] in #36698
fix(model_prices): sync the Groq registry with Groq's docs by @devin-ai-integration[bot] in #36664
fix(router): let untagged requests bypass a tagged pre-routing strategy on shared model names by @mateo-berri in #36627
fix(spend): stop losing spend log rows when a flush is cancelled by @devin-ai-integration[bot] in #34826
docs(claude): drop the @ prefix from the PR template path by @devin-ai-integration[bot] in #36726
fix(langfuse): emit otel trace version and release on the keys langfuse v4 reads by @yucheng-berri in #36702
test(interactions): follow Google spec drift replacing Turn with typed steps by @mateo-berri in #36730
refactor(ui): migrate team detail controls to shadcn by @yuneng-berri in #36695
refactor(ui): migrate guardrail and duration controls to shadcn by @yuneng-berri in #36693
refactor(ui): migrate guardrails-monitor, projects, logs to shadcn by @yuneng-berri in #34606
refactor(ui): migrate search and user controls to shadcn by @yuneng-berri in #36694
fix(guardrails): scan and re-emit raw Anthropic SSE streams in the bedrock post-call hook by @yucheng-berri in #36598
fix(helm): render nodeSelector on the migrations job by @yuneng-berri in #36747
fix(langfuse): coerce header-sourced mask and trace-update steering values by @yucheng-berri in #36740
refactor(ui): migrate usage tables to shared DataTable by @yuneng-berri in #36707
refactor(ui): migrate guardrails monitor table to shared DataTable by @yuneng-berri in #36709
refactor(ui): migrate guardrails content tables to shared DataTable by @yuneng-berri in #36708
feat(gemini): day-0 pricing for gemini-3.7-flash by @mateo-berri in #36792
ci: promote staging to main by @mateo-berri in #36725

New Contributors

@HuanQian571 made their first contribution in #35773
@alexshtf made their first contribution in #35669
@vairodp made their first contribution in #33490
@fancybear-dev made their first contribution in #36196
@daleselaji-dev made their first contribution in #36160
@eugene-yao-zocdoc made their first contribution in #34290
@geraint0923 made their first contribution in #30817
@william-xue made their first contribution in #36529
@dcadenas made their first contribution in #32536
@atomic made their first contribution in #34177
@Praveen11558 made their first contribution in #30952

Full Changelog: v1.97.0-rc.1...v1.98.0-dev.2