BerriAI/litellm v1.98.0-rc.1 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.98.0-rc.1

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.98.0-rc.1/cosign.pub \
  ghcr.io/berriai/litellm:v1.98.0-rc.1

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

fix(bedrock): drop toolSpec.strict for Claude Sonnet 5 on Converse by @kr0k in #33196
build(deps): bump nanoid to 3.3.18 to clear osv-scan by @devin-ai-integration[bot] in #36787
fix(router): stop scoring system prompt text for code/technical complexity by @tin-berri in #36721
feat(complexity_router): calibrate the classifier rubric with worked examples, selectable per router by @tin-berri in #36578
fix(interactions): map step and turn history to Responses API roles and content types by @mateo-berri in #36733
fix(ui): restore playground model filtering by endpoint by @mubashir1osmani in #36130
fix(proxy/batches): stop forwarding custom_llm_provider twice in list and cancel by @anxkhn in #32813
refactor(ui): migrate TokenFlow and JsonViewer to shadcn by @yuneng-berri in #36735
feat: pre-adoption shadow eval for the auto-router (blind pairwise judge, derived state) by @tin-berri in #36587
refactor(ui): migrate SimpleMessageBlock and SimpleToolCallBlock to shadcn by @yuneng-berri in #36737
refactor(ui): migrate HistoryTree and CollapsibleMessage to shadcn by @yuneng-berri in #36738
refactor: replace Any with precise types across responses, proxy, and llms modules by @mateo-berri in #36763
refactor(ui): migrate TruncatedValue and OutputCard to shadcn by @yuneng-berri in #36739
refactor(ui): migrate SectionHeader and ToolsSection to shadcn by @yuneng-berri in #36793
feat(ui): migrate playground chat controls to shadcn by @mubashir1osmani in #36129
feat(xai): day-0 pricing for grok-4.6 by @mateo-berri in #36805
feat(ui): highlight Auto Router in the navbar announcement by @devin-ai-integration[bot] in #36315
test(e2e): assert the model allow-list permits, not only denies by @yuneng-berri in #36823
fix(proxy): tolerate a concurrent creator when creating spend views by @yuneng-berri in #36824
fix(proxy): honor explicit null budget_duration on team and key create + clearable UI dropdowns by @ryan-crabbe-berri in #36699
feat(model_prices): add meta/muse-spark-1.2 and its contributor tier by @mateo-berri in #36717
fix(auth): carry team grants in lite login session tokens by @yassin-berriai in #36826
feat(ui): show provider prompt cache tokens in chat response metrics by @yassin-berriai in #36827
fix(auth): stop the team fallback from widening model access by @yassin-berriai in #36837
fix(proxy/team): resolve member_delete cleanup by user id, not the addressed email by @yassin-berriai in #36839
fix(cli): launch agents as a child process on Windows by @yassin-berriai in #36822
feat(ui): shadow evals tab beside auto-router usage by @tin-berri in #36588
feat(cli): make the hidden lite command list configurable by @yassin-berriai in #36816
feat(azure_ai): add Fireworks FW model pricing on Azure AI Foundry by @emerzon in #35613
fix: enable xhigh reasoning support for gpt-5.4-mini models by @emerzon in #26909
feat(azure-ai): add Grok 4.3 model metadata by @emerzon in #27932
feat(ui): render request metrics on the /ui/chat surface by @yassin-berriai in #36845
fix(ui): stop a deselected MCP server keeping its grant on a virtual key by @yassin-berriai in #36840
fix(team): sweep dangling team references and cache on team delete by @yassin-berriai in #36819
fix(mcp): resolve admin OAuth sessions from any worker via DB-backed drafts by @yassin-berriai in #36844
refactor(ui): migrate usage to shadcn by @yuneng-berri in #36834
refactor(ui): migrate guardrails-monitor to shadcn by @yuneng-berri in #36838
refactor(ui): migrate playground to shadcn by @yuneng-berri in #36847
refactor(ui): migrate guardrails to shadcn by @yuneng-berri in #36832
fix(batches): stop uncostable batches from starving the cost poll page by @devin-ai-integration[bot] in #36714
perf(spend-logs): bound retention cleanup so one run cannot saturate the database by @yassin-berriai in #36594
fix(proxy): fail config load when a callbacks entry is not dispatchable by @yassin-berriai in #36858
fix(bedrock): hoist custom.defer_loading before dropping custom on invoke tools by @yassin-berriai in #36855
fix(access groups): sync assigned_key_ids from the key write paths by @yassin-berriai in #36843
fix(mcp): expose client HTTP headers to logging callbacks and hooks by @devin-ai-integration[bot] in #36724
fix(ptu): stop per-token billing on a PTU-configured deployment by @yucheng-berri in #36829
fix(ui): add nvidia riva to the model provider list by @devin-ai-integration[bot] in #36769
fix(scripts): end make check with a ran/skipped summary and verdict by @mateo-berri in #36864
fix(proxy): track spend for OpenAI passthrough /v1/embeddings by @lostmartian in #36660
test(proxy): stop monkeypatch.undo re-planting fixture-mocked prisma_client by @mateo-berri in #36872
fix(access groups): sync assigned_team_ids from the team write paths by @yassin-berriai in #36825
ci: drop the CircleCI ui_build and ui_unit_tests jobs by @yuneng-berri in #36893
fix(langfuse)!: source the emitted metadata blob from StandardLoggingPayload by @yucheng-berri in #36744
refactor(ui): migrate Navbar off antd to shadcn by @yuneng-berri in #36902
refactor(ui): migrate log details drawer off antd to shadcn by @yuneng-berri in #36904
refactor(ui): migrate AI Hub off antd and tremor to shadcn by @yuneng-berri in #36908
refactor(ui): move the shared dropdowns and selectors onto shadcn primitives by @yuneng-berri in #36924
refactor(ui): move the root-level dashboard components onto shadcn primitives by @yuneng-berri in #36927
refactor(ui): move the settings page and bulk user invite onto shadcn primitives by @yuneng-berri in #36936
refactor(ui): move the cost tracking components onto shadcn primitives by @yuneng-berri in #36955
ci: drop the duplicate proxy_unit_tests letter-shard workflow by @yuneng-berri in #36866
refactor(ui): migrate shared common_components off antd and tremor by @yuneng-berri in #36910
refactor(ui): migrate key info and permissions views off antd and tremor by @yuneng-berri in #36913
feat(proxy): serve Anthropic-native /v1/models for Claude Code gateway discovery by @Ar-maan05 in #35455
refactor(ui): migrate router settings and shared badges off antd and tremor by @yuneng-berri in #36915
refactor(ui): move the model hub and model select onto shadcn primitives by @yuneng-berri in #36918
fix(ui): keep the cost tracking removal confirmation open until it settles by @yuneng-berri in #36960
refactor(ui): declare DateRangePickerValue locally instead of importing it from tremor by @yuneng-berri in #36962
fix(main): an explicit provider outranks a known OpenAI model name by @FahimaGold in #36800
fix(exception_mapping): bare 429 in an error body no longer outranks the status code by @FahimaGold in #36705
refactor(ui): move MCP permission panels onto shadcn primitives by @yuneng-berri in #36964
refactor(ui): migrate ten small dashboard files off antd and tremor by @yuneng-berri in #36966
fix(proxy): force prisma recreate on postgres cached-plan error by @devin-ai-integration[bot] in #36428
fix(transcription): stop a zero output rate from zeroing transcription cost by @hMED22 in #36914
fix(langfuse): restrict trace steering keys to real langfuse trace fields by @yucheng-berri in #36862
Revert "fix(auth): stop the team fallback from widening model access" (#36837) by @yuneng-berri in #36982
fix(ui): show zeroed auto-router usage stats when a window has no sessions by @tin-berri in #36868
fix(mcp): keep admin-entered oauth endpoints in management reads by @devin-ai-integration[bot] in #36888
fix(ui): distinguish hosted and local vLLM in the provider dropdown by @mateo-berri in #36974
fix(openai,azure): return a length-truncated 200 when the output budget fits no token by @yassin-berriai in #36859
fix(proxy): always emit the Anthropic /v1/models token limits, null when unknown by @yassin-berriai in #36961
feat(helm): add startupProbe and hpa.behavior to the componentized chart by @Louis-Vauterin in #36382
fix(proxy): serve aggregate MCP endpoint on bare /mcp instead of 307-redirecting by @tin-berri in #34845
feat(shadow_eval): add reverse-direction shadow eval jobs by @tin-berri in #36865
fix(proxy): requeue Redis spend buffer transactions when the DB commit fails by @devin-ai-integration[bot] in #33881
feat(search): add Nimble as a search provider by @ilchemla in #36347
fix(mcp): drop caller host and configured upstream headers from logged metadata by @yucheng-berri in #36901
fix(azure_ai): recognize real Search doc endpoints so teams can read/write via passthrough by @devin-ai-integration[bot] in #36798
fix(anthropic): aggregate 5m/1h cache-write split across iterations path by @devin-ai-integration[bot] in #34860
fix(anthropic cost): apply regional geo uplift to cached tokens by @devin-ai-integration[bot] in #34850
fix(ui): match the MCP servers count badge to its sibling permission badges by @yuneng-berri in #36984
fix(batches): mark terminal batch with no output file as processed in CheckBatchCost by @devin-ai-integration[bot] in #35360
fix(caching): cache anthropic /v1/messages responses, including streaming by @devin-ai-integration[bot] in #34581
fix(anthropic_messages): make tool_result images visible to OpenAI-compatible providers by @hMED22 in #34462
feat(fireworks_ai): translate NIM/vLLM extra params to Fireworks-native args by @milesadkins in #35969
fix(ui): stop the models tab strip from scrolling vertically by @yuneng-berri in #36993
fix(ui): anchor chips-combobox popups to the field instead of the inner input by @yuneng-berri in #36995
feat(proxy): per-component response cost headers by @erensh27 in #36965
fix(cost): track OpenAI/Azure web search tool cost per call by @devin-ai-integration[bot] in #35286
fix(bedrock): resolve aliases in batch file records by @daleselaji-dev in #36159
fix: report real token usage on guardrail-blocked /v1/responses replies by @guptaishaan in #36907
fix(proxy): requeue spend logs when the DB write fails with a transport error by @devin-ai-integration[bot] in #36716
fix(cost): tiered pricing supports cache creation cost and is all-or-nothing by @devin-ai-integration[bot] in #36720
fix(vertex_ai): translate /v1/embeddings batch rows to the Gemini embedding shape by @devin-ai-integration[bot] in #35092
docs(claude): require ReadOnly on every TypedDict field (LIT012) by @devin-ai-integration[bot] in #37005
refactor(ui): migrate access group create modal to RHF + zod + shadcn by @ryan-crabbe-berri in #37033
refactor(ui): re-sync badge and skeleton onto the base-vega shadcn style by @yuneng-berri in #36991
feat(ui): link user detail team names to team pages by @ryan-crabbe-berri in #37022
fix(model_prices): correct DeepSeek V4 max output tokens by @devin-ai-integration[bot] in #36925
fix(ui): rename models table Status column to Source by @ryan-crabbe-berri in #37021
chore: bump litellm-enterprise 0.1.55 -> 0.1.56, litellm-proxy-extras 0.4.85 -> 0.4.86 by @yuneng-berri in #37045
feat(proxy): gate the Global Control Plane worker registry on an enterprise license by @yassin-berriai in #36996
fix(model_prices): add gemini 3.1 flash tts preview and legacy OpenAI shutdown dates by @devin-ai-integration[bot] in #36788
fix(panw_prisma_airs): surface scan_id on allowed requests by @devin-ai-integration[bot] in #37037
fix(model_map): flag native structured outputs on Anthropic-direct claude-sonnet-5 and claude-haiku-4-5 by @anmolg1997 in #35930
fix(router): stop get_router_model_info from wiping cached pricing by @devin-ai-integration[bot] in #36985
fix(redis): unwrap decorated __init__s when deriving the from_url kwargs allowlist by @anmolg1997 in #36654
fix(proxy): reserve the larger declared output budget for TPM limits by @yassin-berriai in #37001
fix(databricks): surface provider usage, including prompt-cache counts, in streaming chunks by @pokepoke81 in #36943
fix(spend): give a batch's cost row a primary key of its own by @marty-sullivan in #36876
feat: shadow eval samples /v1/messages and /v1/responses traffic by @tin-berri in #36830
fix(ptu): stop a PTU deployment billing for grounded search by @yucheng-berri in #37043
fix(fireworks_ai): support router slugs via routers/ prefix by @heathriel in #34257
fix(bedrock): register managed-batch litellm_params so they stop leaking to the provider (internal copy of #36633) by @mateo-berri in #37048
fix(bedrock): resolve the managed-batch output bucket on every path that reads it by @mateo-berri in #37047
fix(bedrock): resolve the managed-batch output bucket on every path that reads it by @marty-sullivan in #36634
feat(scripts): queue heavy gates behind a machine-wide slot lock by @mateo-berri in #36988
feat(mcp): scope gateway session bearers to the RFC 8707 resource by @tin-berri in #35045
feat(ui): direction picker and reverse-mode display for shadow evals by @tin-berri in #36994
fix(guardrails): return the full PANW AIRS scan response on blocked requests by @devin-ai-integration[bot] in #37036
fix(passthrough): stop forwarding client Accept-Encoding upstream by @mateo-berri in #37058
fix(batches): account a managed batch's cost exactly once by @mateo-berri in #37050
fix(panw_prisma_airs): scan tool call args as plain text, not a tool_event by @devin-ai-integration[bot] in #37038
feat(lint): exempt TypedDict-annotated dict literals from LIT002 by @mateo-berri in #36869
docs(claude): tell agents to let heavy gates queue for machine-wide slots by @devin-ai-integration[bot] in #37057
test: unstick the suites CircleCI is failing on by @yuneng-berri in #37059
docs(github): proof-of-fix section shows only the latest run as Before/After with nested cases by @mateo-berri in #37063
test(e2e): assert provider error shape instead of pinned prose by @yuneng-berri in #37065
fix(ui): de-duplicate the reset budget option and polish shadcn surfaces by @yuneng-berri in #37010
chore: rebuild Admin UI bundle from litellm_internal_staging by @yuneng-berri in #37066
test(e2e/ui): assert the log drawer chevrons by their lucide classes by @yuneng-berri in #37069
chore(ci): promote internal staging to main by @yuneng-berri in #37042

New Contributors

@kr0k made their first contribution in #33196
@anxkhn made their first contribution in #32813
@lostmartian made their first contribution in #36660
@FahimaGold made their first contribution in #36800
@Louis-Vauterin made their first contribution in #36382
@ilchemla made their first contribution in #36347
@milesadkins made their first contribution in #35969
@erensh27 made their first contribution in #36965
@guptaishaan made their first contribution in #36907
@pokepoke81 made their first contribution in #36943
@heathriel made their first contribution in #34257

Full Changelog: v1.98.0-dev.2...v1.98.0-rc.1