BerriAI/litellm v1.99.0-dev.1 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.99.0-dev.1

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.99.0-dev.1/cosign.pub \
  ghcr.io/berriai/litellm:v1.99.0-dev.1

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

chore(typing): drop 1.3k basedpyright errors across 30 Any hotspot files by @mateo-berri in #37073
fix(proxy): register WebSocket passthrough for OpenAI prefixes by @LHMQ878 in #36151
fix(bedrock): report uploaded size in the FileObject returned by managed batch uploads by @devin-ai-integration[bot] in #36392
fix(batches): support AWS Bedrock batch cancellation via StopModelInvocationJob by @ArjunPakhan in #34087
feat: Async Rust OCR Bridge and MCP OAuth UI Restore by @ArjunPakhan in #31453
fix(batches): don't crash logging when a completed batch has no output file by @MUSE-CODE-SPACE in #34067
fix(UI): add default model pin to complexity router UI by @tin-berri in #36615
feat(ui): add Lite mixed-provider auto-router preset by @tin-berri in #37068
feat(ui): link key info header to its user, creator, team, and organization by @ryan-crabbe-berri in #37187
fix(guardrails): scan text on /guardrails/apply_guardrail for Azure Content Safety by @yucheng-berri in #36894
feat(bedrock): forward LiteLLM identity and metadata into Bedrock requestMetadata by @yassin-berriai in #36861
fix(azure): rename max_tokens to max_completion_tokens for gpt-5-chat deployments by @yassin-berriai in #36857
fix(bedrock): preserve cache token usage when invocationMetrics replace the usage block by @brian5021 in #36878
fix(proxy): registry caches stop per-request tag and end-user Postgres reads in auth by @ryan-crabbe-berri in #36801
test(e2e): replay a real tool-search assistant turn back to Bedrock Invoke by @yassin-berriai in #36856
fix(proxy): return 400 naming the missing required param on POST /v1/batches by @mateo-berri in #37199
fix(ci): bump sqlparse to 0.6.0 to resolve osv-scan CVEs by @mateo-berri in #37200
fix(ui): stop pairing key spend with the team budget when a key has no budget by @ryan-crabbe-berri in #37196
fix(guardrails): record MCP tool guardrail evaluations and blocks in … by @Scott-Wilson-ZocDoc in #36978
fix(proxy): return 400 for non-object metadata and litellm_metadata instead of silent drop or 500 by @mateo-berri in #37203
fix(anthropic): preserve optional Responses tool properties by @Scott-Wilson-ZocDoc in #36979
feat(ui): add user ID request log filter by @daniel-meismer-zocdoc in #36781
fix(anthropic): stop emitting empty thinking blocks on the Responses adapter by @Scott-Wilson-ZocDoc in #36033
fix(ui): make per-user usage filter searchable by @daniel-meismer-zocdoc in #36790
refactor(ui): decouple bulk invite from the invite user button by @yuneng-berri in #37061
fix(helm): bound the migrations Job so a blocked migration cannot stall the release by @yuneng-berri in #36975
feat(proxy): let USE_V2_MIGRATION_RESOLVER select the v2 migration resolver by @yuneng-berri in #36258
fix(mcp): scope authorization server issuer by @irosh-colombage-ZocDoc2 in #36482
fix(responses): unwrap object-form tool_choice before calling the Responses API by @Scott-Wilson-ZocDoc in #36032
test(ui): query antd controls accessibly instead of by internal CSS class by @yuneng-berri in #37014
fix(proxy): bill cancelled and failed batches that still produced an output file by @mateo-berri in #37205
fix(bedrock): read batch usage by payload shape, not by provider name by @marty-sullivan in #37078
fix(ui): self-contained searchable user filter on the Usage page by @ryan-crabbe-berri in #37206
revert: don't fix mcp scope authorization server issuer by @mateo-berri in #37220
fix(mcp): scope authorization server issuer for named MCP servers by @yucheng-berri in #37204
test(ui): gate dashboard test assertions with testing-library and jest-dom rules by @yuneng-berri in #37018
feat(shadow-eval): name the shadowed key in job responses and the UI headline by @tin-berri in #37221
test(ui): assert what collaborators are called with, not merely that they were by @yuneng-berri in #37019
fix(logging): stop deepcopying results redaction cannot redact by @marty-sullivan in #36638
fix(gemini): price gemini 3.6 flash at Google's introductory rates on every service tier by @devin-ai-integration[bot] in #37197
perf(guardrails): stop sending the conversation twice in the noma v2 payload by @itaimodi in #36764
fix(streaming): track provider-reported cost when caller omits include_usage by @devin-ai-integration[bot] in #35013
fix: stop rust flag from leaking into upstream provider request bodies by @mateo-berri in #37218
fix(proxy): return 404 instead of 500 for unresolvable batch and file ids on /v1/batches by @mateo-berri in #37201
fix(bedrock): validate file-content retrieval against the configured output bucket (#26335) by @kingdoooo in #31435
fix(proxy): reject out-of-range limit on GET /v1/batches with OpenAI-parity 400 by @mateo-berri in #37198
fix(batches): price a retrieved batch from its deployment's model and rates (internal copy of #37077) by @mateo-berri in #37219
feat(ocr): return Azure Document Intelligence's native payload from /v1/ocr via req_format=native by @devin-ai-integration[bot] in #37194
fix(anthropic): fold guardrail-modified leading system rows into top-level system param by @mateo-berri in #37231
fix(shadow_eval): copy messages before router call and raise judge output cap by @tin-berri in #37232
feat(proxy): add Amazon Comprehend Medical passthrough provider by @mateo-berri in #37229
test(ui): settle the in-flight search before the loading tests end by @yuneng-berri in #37227
test(cli): use example.com placeholder host in base-url trailing slash test by @ryan-crabbe-berri in #37240
feat(complexity_router): operator-defined tier sets for the LLM classifier by @tin-berri in #37226
feat(ui): configure the auto router's heuristic scorer from the Admin UI by @tin-berri in #37216
fix(shadow_eval): drop unused judge reasoning field and salvage truncated verdicts by @tin-berri in #37239
feat(proxy): proactive model deprecation alerts and /model/deprecations endpoint by @mateo-berri in #26900
refactor(ui): move dashboard toasts from antd message/notification onto sonner by @ryan-crabbe-berri in #37207
feat(guardrails): track bedrock guardrail usage units per invocation by @mateo-berri in #37225
fix(proxy): strip callback credentials from the auth object stamped into request metadata by @yucheng-berri in #37233
fix(guardrails): retry usage upserts only on connection errors by @mateo-berri in #37247
fix(mcp): oauth discovery must not cause outages by @daniel-meismer-zocdoc in #36599
test(ui): await the playground model combobox before clicking it by @yuneng-berri in #36850
refactor(ui): migrate budget and skill forms to react-hook-form and shadcn by @yuneng-berri in #37262
refactor(ui): migrate tag and memory forms to react-hook-form and shadcn by @yuneng-berri in #37266
feat(complexity_router): plan-mode tier floor for coding-agent clients by @tin-berri in #37230
refactor(ui): codemod every toast call site onto lib/toast and delete the antd-era facades by @ryan-crabbe-berri in #37253
feat(proxy): add /team/daily/activity/aggregated and switch the Usage team tab to it by @ryan-crabbe-berri in #36562
refactor(ui): migrate user, logging and policy forms to react-hook-form and shadcn by @yuneng-berri in #37303
refactor(ui): migrate user, policy, and margin forms to shadcn by @yuneng-berri in #37305
refactor(ui): migrate the regenerate key and team member forms to react-hook-form and shadcn by @yuneng-berri in #37300
refactor(ui): migrate CloudZero and cost tracking forms to react-hook-form and shadcn by @yuneng-berri in #37312
refactor(ui): migrate auto router and credential forms to react-hook-form and shadcn by @yuneng-berri in #37304
refactor(ui): migrate guardrail and vector store forms to react-hook-form and shadcn by @yuneng-berri in #37306
refactor(ui): migrate prompt, UI access, plugin and MCP filter forms to react-hook-form and shadcn by @yuneng-berri in #37297
refactor(ui): drop the unreachable user edit modal by @yuneng-berri in #37327
fix(router): route Responses API input through the auto-router by @mateo-berri in #37333
refactor(ui): migrate the login, onboarding and search tool forms to react-hook-form and shadcn by @yuneng-berri in #37334
feat(ui): plan-mode override tier in the auto-router create and edit forms by @tin-berri in #37319
refactor(ui): retire the tremor date range picker in favour of the shared advanced picker by @ryan-crabbe-berri in #37302
fix(proxy): forward Bedrock event-stream content-type on unbuffered passthrough by @devin-ai-integration[bot] in #33767
fix(azure_ai): strip non-OpenAI-spec message fields before request by @ayaangazali in #34445
fix(proxy): stop leaking the client_side_timeout marker to providers by @mateo-berri in #37346
refactor(ui): migrate the caching, cost tracking, alerting and user detail forms to react-hook-form and shadcn by @yuneng-berri in #37350
refactor(ui): migrate pass-through, project and access group forms to react-hook-form and shadcn by @yuneng-berri in #37354
refactor(ui): migrate the vector store creation form to shadcn by @yuneng-berri in #37353
refactor(ui): move the MCP server forms and detail tabs off tremor by @ryan-crabbe-berri in #37329
feat(complexity_router): custom classifier plugins via classifier_type 'custom' by @tin-berri in #37249
fix(fireworks): skip accounts/ rewrite for FW-* Foundry deployment ids by @bruno-olivia in #37242
fix(advisor): resolve the advisor sub-call through the proxy router by @devin-ai-integration[bot] in #36246
fix(router): forward target_model_names on file uploads to litellm_proxy deployments by @shivamrawat1 in #36240
refactor(ui): move the internal user detail view off tremor by @ryan-crabbe-berri in #37309
fix(responses): strip the responses/ routing prefix on the Responses API path by @mateo-berri in #37345
fix(main): forward store and prompt_cache_key params on chat completions by @Sujithr07 in #33195
refactor(ui): migrate the model settings and credential rotation modals to react-hook-form and shadcn by @yuneng-berri in #37342
refactor(ui): move the shared key form controls off antd onto shadcn by @yuneng-berri in #37348
refactor(ui): move the tag and vector store views off tremor by @ryan-crabbe-berri in #37311
refactor(ui): migrate SSO, SCIM and vault forms to react-hook-form and shadcn by @yuneng-berri in #37347
test(ui): cover the edit project modal's required-field validation by @yuneng-berri in #37363
refactor(ui): migrate the guardrail forms to react-hook-form and shadcn by @yuneng-berri in #37364
refactor(ui): migrate agent forms to react-hook-form and shadcn by @yuneng-berri in #37357
refactor(ui): migrate the MCP per-user env vars, toolset and tool arguments forms to react-hook-form and shadcn by @yuneng-berri in #37349
fix(anthropic): emit tool_use content_block_start without awaiting the next chunk by @yassin-berriai in #37310
fix(proxy): send SSE keepalives while a slow upstream is still silent by @yassin-berriai in #37322
fix(proxy): let org admins view their organization's usage by @yassin-berriai in #37235
feat(vector_stores): add Valkey as a managed vector store provider by @ryan-crabbe-berri in #37002
refactor(ui): move the virtual key create and edit forms off tremor by @ryan-crabbe-berri in #37324
refactor(ui): move the add model and credential forms off tremor by @ryan-crabbe-berri in #37325
feat(team-callbacks): add DELETE /team/{team_id}/callback/{callback_name} by @yucheng-berri in #37331
refactor(ui): move the teams page and team detail views off tremor by @ryan-crabbe-berri in #37317
fix(cost_calculator): recognize the ultrafast service tier in cost calculation by @devin-ai-integration[bot] in #37355
refactor(ui): move the cache settings and playground model selector off tremor by @ryan-crabbe-berri in #37323
feat(guardrails): count bedrock guardrail cost against spend and budgets by @mateo-berri in #37362
refactor(ui): move the admin, SSO, SCIM, alerting and fallback forms off tremor by @ryan-crabbe-berri in #37315
test(ui): raise vitest test and hook timeouts for CI headroom by @yuneng-berri in #37370
fix(caching): truncate semantic cache embedding input, send extra_body top-level by @mateo-berri in #37367
fix(guardrails): cap the date window accepted by /guardrails/usage endpoints by @mateo-berri in #37380
fix(ui): show select labels on the trigger instead of raw values by @yuneng-berri in #37372
refactor(ui): move the team member search modal off antd Form by @yuneng-berri in #37383
refactor(ui): move the model alias manager onto design tokens and shadcn controls by @yuneng-berri in #37376
refactor(ui): move the MCP tool test form off antd by @yuneng-berri in #37381
refactor(ui): move the model info view and pass-through endpoint forms off tremor by @ryan-crabbe-berri in #37308
fix(otel): bound and shut down credential-scoped tracer providers by @yucheng-berri in #36591
fix(proxy): send SSE keepalives on assistants runs and A2A streams by @yassin-berriai in #37368
test(anthropic): pin one content_block_stop per tool_use block on the Responses adapter by @mateo-berri in #37356
refactor(ui): move the agent, guardrail, prompt, policy and skill forms off tremor by @ryan-crabbe-berri in #37320
fix(guardrails): requeue usage rollup rows dropped after retry exhaustion by @mateo-berri in #37387
feat(proxy): add project-level ITPM and OTPM quotas by @shivijain2323 in #35110
feat(bedrock): add a config toggle to disable agent-runtime pass-through by @yuneng-berri in #37386
fix(mcp): attach per-user BYOK credential when listing tools for non-oauth2 auth types by @devin-ai-integration[bot] in #34787
feat(ui): add success, warning and info status tokens by @yuneng-berri in #37393
refactor(ui): drop @tremor/react and the theming scaffolding it needed by @ryan-crabbe-berri in #37394
refactor(ui): move the model info edit form off antd Form by @yuneng-berri in #37392
fix(vector_stores): stop leaking stored credentials in direct search debug logs by @ryan-crabbe-berri in #37373
fix(logging): close three secret-leak paths in verbose logging by @ryan-crabbe-berri in #37391
chore: bump litellm-enterprise 0.1.56 -> 0.1.57, litellm-proxy-extras 0.4.86 -> 0.4.87, litellm 1.98.0 -> 1.99.0 by @yuneng-berri in #37395
fix(mcp): bind tool existence check to the selected server by @mateo-berri in #37388
fix(mcp): serve token-forwarding servers when oauth discovery fails by @tin-berri in #37399
fix(databricks): add cost map entries for 14 newer Databricks models by @epistoteles in #28501
refactor(ui): style the logging settings from semantic tokens by @yuneng-berri in #37385
feat(tinyfish): surface response headers + top-level response extras by @ChenluJi in #32448
refactor(ui): codemod the antd Tooltips outside form files onto the shadcn atom by @ryan-crabbe-berri in #37402
test(ocr): update Azure DI supported-params assertion for req_format by @mateo-berri in #37419
fix(proxy): return no rows when the aggregated activity entity filter is empty by @yuneng-berri in #37414
test: build redaction and batch limiter fixtures the way production does by @yuneng-berri in #37416
test: allow protocol-constrained pass-through routes to declare fewer methods by @yuneng-berri in #37415
test(ui): pin the MCP server edit save payload before the form migration by @yuneng-berri in #37404
test(ui): characterize the create key form payload contract by @yuneng-berri in #37405
refactor(ui): migrate the key edit form off Ant Design onto react-hook-form by @yuneng-berri in #37398
test(ui): repoint the e2e locators at the post-antd form controls by @yuneng-berri in #37421
test: point the live gemini and groq conformance suites at models that still exist by @yuneng-berri in #37422
refactor(ui): extract the create-key payload builder out of create_key_button by @yuneng-berri in #37397
fix(types): map nested prompt_tokens_details.cache_creation_input_tokens to cache_write_tokens by @devin-ai-integration[bot] in #37377
fix(router): honor key-level tag filtering in pre-routing and pin auto-router e2e regressions by @mateo-berri in #37366
feat(otel): attribute Prisma database spans to PostgreSQL instead of localhost by @yucheng-berri in #36595
fix(bedrock): degrade gracefully on malformed tool-call arguments by @devin-ai-integration[bot] in #33842
test: move the remaining live groq call sites off the retired llama models by @yuneng-berri in #37426
fix(ui): highlight the first member search match so Enter picks it by @yuneng-berri in #37429
chore(ci): promote internal staging to main by @yuneng-berri in #37400

New Contributors

@LHMQ878 made their first contribution in #36151
@ArjunPakhan made their first contribution in #34087
@MUSE-CODE-SPACE made their first contribution in #34067
@brian5021 made their first contribution in #36878
@Scott-Wilson-ZocDoc made their first contribution in #36978
@irosh-colombage-ZocDoc2 made their first contribution in #36482
@itaimodi made their first contribution in #36764
@ayaangazali made their first contribution in #34445
@bruno-olivia made their first contribution in #37242
@Sujithr07 made their first contribution in #33195
@shivijain2323 made their first contribution in #35110
@ChenluJi made their first contribution in #32448

Full Changelog: v1.98.0-rc.1...v1.99.0-dev.1