BerriAI/litellm v1.99.0 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.99.0

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.99.0/cosign.pub \
  ghcr.io/berriai/litellm:v1.99.0

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

chore(typing): drop 1.3k basedpyright errors across 30 Any hotspot files by @mateo-berri in #37073
fix(proxy): register WebSocket passthrough for OpenAI prefixes by @LHMQ878 in #36151
fix(bedrock): report uploaded size in the FileObject returned by managed batch uploads by @devin-ai-integration[bot] in #36392
fix(batches): support AWS Bedrock batch cancellation via StopModelInvocationJob by @ArjunPakhan in #34087
feat: Async Rust OCR Bridge and MCP OAuth UI Restore by @ArjunPakhan in #31453
fix(batches): don't crash logging when a completed batch has no output file by @MUSE-CODE-SPACE in #34067
fix(UI): add default model pin to complexity router UI by @tin-berri in #36615
feat(ui): add Lite mixed-provider auto-router preset by @tin-berri in #37068
feat(ui): link key info header to its user, creator, team, and organization by @ryan-crabbe-berri in #37187
fix(guardrails): scan text on /guardrails/apply_guardrail for Azure Content Safety by @yucheng-berri in #36894
feat(bedrock): forward LiteLLM identity and metadata into Bedrock requestMetadata by @yassin-berriai in #36861
fix(azure): rename max_tokens to max_completion_tokens for gpt-5-chat deployments by @yassin-berriai in #36857
fix(bedrock): preserve cache token usage when invocationMetrics replace the usage block by @brian5021 in #36878
fix(proxy): registry caches stop per-request tag and end-user Postgres reads in auth by @ryan-crabbe-berri in #36801
test(e2e): replay a real tool-search assistant turn back to Bedrock Invoke by @yassin-berriai in #36856
fix(proxy): return 400 naming the missing required param on POST /v1/batches by @mateo-berri in #37199
fix(ci): bump sqlparse to 0.6.0 to resolve osv-scan CVEs by @mateo-berri in #37200
fix(ui): stop pairing key spend with the team budget when a key has no budget by @ryan-crabbe-berri in #37196
fix(guardrails): record MCP tool guardrail evaluations and blocks in … by @Scott-Wilson-ZocDoc in #36978
fix(proxy): return 400 for non-object metadata and litellm_metadata instead of silent drop or 500 by @mateo-berri in #37203
fix(anthropic): preserve optional Responses tool properties by @Scott-Wilson-ZocDoc in #36979
feat(ui): add user ID request log filter by @daniel-meismer-zocdoc in #36781
fix(anthropic): stop emitting empty thinking blocks on the Responses adapter by @Scott-Wilson-ZocDoc in #36033
fix(ui): make per-user usage filter searchable by @daniel-meismer-zocdoc in #36790
refactor(ui): decouple bulk invite from the invite user button by @yuneng-berri in #37061
fix(helm): bound the migrations Job so a blocked migration cannot stall the release by @yuneng-berri in #36975
feat(proxy): let USE_V2_MIGRATION_RESOLVER select the v2 migration resolver by @yuneng-berri in #36258
fix(mcp): scope authorization server issuer by @irosh-colombage-ZocDoc2 in #36482
fix(responses): unwrap object-form tool_choice before calling the Responses API by @Scott-Wilson-ZocDoc in #36032
test(ui): query antd controls accessibly instead of by internal CSS class by @yuneng-berri in #37014
fix(proxy): bill cancelled and failed batches that still produced an output file by @mateo-berri in #37205
fix(bedrock): read batch usage by payload shape, not by provider name by @marty-sullivan in #37078
fix(ui): self-contained searchable user filter on the Usage page by @ryan-crabbe-berri in #37206
revert: don't fix mcp scope authorization server issuer by @mateo-berri in #37220
fix(mcp): scope authorization server issuer for named MCP servers by @yucheng-berri in #37204
test(ui): gate dashboard test assertions with testing-library and jest-dom rules by @yuneng-berri in #37018
feat(shadow-eval): name the shadowed key in job responses and the UI headline by @tin-berri in #37221
test(ui): assert what collaborators are called with, not merely that they were by @yuneng-berri in #37019
fix(logging): stop deepcopying results redaction cannot redact by @marty-sullivan in #36638
fix(gemini): price gemini 3.6 flash at Google's introductory rates on every service tier by @devin-ai-integration[bot] in #37197
perf(guardrails): stop sending the conversation twice in the noma v2 payload by @itaimodi in #36764
fix(streaming): track provider-reported cost when caller omits include_usage by @devin-ai-integration[bot] in #35013
fix: stop rust flag from leaking into upstream provider request bodies by @mateo-berri in #37218
fix(proxy): return 404 instead of 500 for unresolvable batch and file ids on /v1/batches by @mateo-berri in #37201
fix(bedrock): validate file-content retrieval against the configured output bucket (#26335) by @kingdoooo in #31435
fix(proxy): reject out-of-range limit on GET /v1/batches with OpenAI-parity 400 by @mateo-berri in #37198
fix(batches): price a retrieved batch from its deployment's model and rates (internal copy of #37077) by @mateo-berri in #37219
feat(ocr): return Azure Document Intelligence's native payload from /v1/ocr via req_format=native by @devin-ai-integration[bot] in #37194
fix(anthropic): fold guardrail-modified leading system rows into top-level system param by @mateo-berri in #37231
fix(shadow_eval): copy messages before router call and raise judge output cap by @tin-berri in #37232
feat(proxy): add Amazon Comprehend Medical passthrough provider by @mateo-berri in #37229
test(ui): settle the in-flight search before the loading tests end by @yuneng-berri in #37227
test(cli): use example.com placeholder host in base-url trailing slash test by @ryan-crabbe-berri in #37240
feat(complexity_router): operator-defined tier sets for the LLM classifier by @tin-berri in #37226
feat(ui): configure the auto router's heuristic scorer from the Admin UI by @tin-berri in #37216
fix(shadow_eval): drop unused judge reasoning field and salvage truncated verdicts by @tin-berri in #37239
feat(proxy): proactive model deprecation alerts and /model/deprecations endpoint by @mateo-berri in #26900
refactor(ui): move dashboard toasts from antd message/notification onto sonner by @ryan-crabbe-berri in #37207
feat(guardrails): track bedrock guardrail usage units per invocation by @mateo-berri in #37225
fix(proxy): strip callback credentials from the auth object stamped into request metadata by @yucheng-berri in #37233
fix(guardrails): retry usage upserts only on connection errors by @mateo-berri in #37247
fix(mcp): oauth discovery must not cause outages by @daniel-meismer-zocdoc in #36599
test(ui): await the playground model combobox before clicking it by @yuneng-berri in #36850
refactor(ui): migrate budget and skill forms to react-hook-form and shadcn by @yuneng-berri in #37262
refactor(ui): migrate tag and memory forms to react-hook-form and shadcn by @yuneng-berri in #37266
feat(complexity_router): plan-mode tier floor for coding-agent clients by @tin-berri in #37230
refactor(ui): codemod every toast call site onto lib/toast and delete the antd-era facades by @ryan-crabbe-berri in #37253
feat(proxy): add /team/daily/activity/aggregated and switch the Usage team tab to it by @ryan-crabbe-berri in #36562
refactor(ui): migrate user, logging and policy forms to react-hook-form and shadcn by @yuneng-berri in #37303
refactor(ui): migrate user, policy, and margin forms to shadcn by @yuneng-berri in #37305
refactor(ui): migrate the regenerate key and team member forms to react-hook-form and shadcn by @yuneng-berri in #37300
refactor(ui): migrate CloudZero and cost tracking forms to react-hook-form and shadcn by @yuneng-berri in #37312
refactor(ui): migrate auto router and credential forms to react-hook-form and shadcn by @yuneng-berri in #37304
refactor(ui): migrate guardrail and vector store forms to react-hook-form and shadcn by @yuneng-berri in #37306
refactor(ui): migrate prompt, UI access, plugin and MCP filter forms to react-hook-form and shadcn by @yuneng-berri in #37297
refactor(ui): drop the unreachable user edit modal by @yuneng-berri in #37327
fix(router): route Responses API input through the auto-router by @mateo-berri in #37333
refactor(ui): migrate the login, onboarding and search tool forms to react-hook-form and shadcn by @yuneng-berri in #37334
feat(ui): plan-mode override tier in the auto-router create and edit forms by @tin-berri in #37319
refactor(ui): retire the tremor date range picker in favour of the shared advanced picker by @ryan-crabbe-berri in #37302
fix(proxy): forward Bedrock event-stream content-type on unbuffered passthrough by @devin-ai-integration[bot] in #33767
fix(azure_ai): strip non-OpenAI-spec message fields before request by @ayaangazali in #34445
fix(proxy): stop leaking the client_side_timeout marker to providers by @mateo-berri in #37346
refactor(ui): migrate the caching, cost tracking, alerting and user detail forms to react-hook-form and shadcn by @yuneng-berri in #37350
refactor(ui): migrate pass-through, project and access group forms to react-hook-form and shadcn by @yuneng-berri in #37354
refactor(ui): migrate the vector store creation form to shadcn by @yuneng-berri in #37353
refactor(ui): move the MCP server forms and detail tabs off tremor by @ryan-crabbe-berri in #37329
feat(complexity_router): custom classifier plugins via classifier_type 'custom' by @tin-berri in #37249
fix(fireworks): skip accounts/ rewrite for FW-* Foundry deployment ids by @bruno-olivia in #37242
fix(advisor): resolve the advisor sub-call through the proxy router by @devin-ai-integration[bot] in #36246
fix(router): forward target_model_names on file uploads to litellm_proxy deployments by @shivamrawat1 in #36240
refactor(ui): move the internal user detail view off tremor by @ryan-crabbe-berri in #37309
fix(responses): strip the responses/ routing prefix on the Responses API path by @mateo-berri in #37345
fix(main): forward store and prompt_cache_key params on chat completions by @Sujithr07 in #33195
refactor(ui): migrate the model settings and credential rotation modals to react-hook-form and shadcn by @yuneng-berri in #37342
refactor(ui): move the shared key form controls off antd onto shadcn by @yuneng-berri in #37348
refactor(ui): move the tag and vector store views off tremor by @ryan-crabbe-berri in #37311
refactor(ui): migrate SSO, SCIM and vault forms to react-hook-form and shadcn by @yuneng-berri in #37347
test(ui): cover the edit project modal's required-field validation by @yuneng-berri in #37363
refactor(ui): migrate the guardrail forms to react-hook-form and shadcn by @yuneng-berri in #37364
refactor(ui): migrate agent forms to react-hook-form and shadcn by @yuneng-berri in #37357
refactor(ui): migrate the MCP per-user env vars, toolset and tool arguments forms to react-hook-form and shadcn by @yuneng-berri in #37349
fix(anthropic): emit tool_use content_block_start without awaiting the next chunk by @yassin-berriai in #37310
fix(proxy): send SSE keepalives while a slow upstream is still silent by @yassin-berriai in #37322
fix(proxy): let org admins view their organization's usage by @yassin-berriai in #37235
feat(vector_stores): add Valkey as a managed vector store provider by @ryan-crabbe-berri in #37002
refactor(ui): move the virtual key create and edit forms off tremor by @ryan-crabbe-berri in #37324
refactor(ui): move the add model and credential forms off tremor by @ryan-crabbe-berri in #37325
feat(team-callbacks): add DELETE /team/{team_id}/callback/{callback_name} by @yucheng-berri in #37331
refactor(ui): move the teams page and team detail views off tremor by @ryan-crabbe-berri in #37317
fix(cost_calculator): recognize the ultrafast service tier in cost calculation by @devin-ai-integration[bot] in #37355
refactor(ui): move the cache settings and playground model selector off tremor by @ryan-crabbe-berri in #37323
feat(guardrails): count bedrock guardrail cost against spend and budgets by @mateo-berri in #37362
refactor(ui): move the admin, SSO, SCIM, alerting and fallback forms off tremor by @ryan-crabbe-berri in #37315
test(ui): raise vitest test and hook timeouts for CI headroom by @yuneng-berri in #37370
fix(caching): truncate semantic cache embedding input, send extra_body top-level by @mateo-berri in #37367
fix(guardrails): cap the date window accepted by /guardrails/usage endpoints by @mateo-berri in #37380
fix(ui): show select labels on the trigger instead of raw values by @yuneng-berri in #37372
refactor(ui): move the team member search modal off antd Form by @yuneng-berri in #37383
refactor(ui): move the model alias manager onto design tokens and shadcn controls by @yuneng-berri in #37376
refactor(ui): move the MCP tool test form off antd by @yuneng-berri in #37381
refactor(ui): move the model info view and pass-through endpoint forms off tremor by @ryan-crabbe-berri in #37308
fix(otel): bound and shut down credential-scoped tracer providers by @yucheng-berri in #36591
fix(proxy): send SSE keepalives on assistants runs and A2A streams by @yassin-berriai in #37368
test(anthropic): pin one content_block_stop per tool_use block on the Responses adapter by @mateo-berri in #37356
refactor(ui): move the agent, guardrail, prompt, policy and skill forms off tremor by @ryan-crabbe-berri in #37320
fix(guardrails): requeue usage rollup rows dropped after retry exhaustion by @mateo-berri in #37387
feat(proxy): add project-level ITPM and OTPM quotas by @shivijain2323 in #35110
feat(bedrock): add a config toggle to disable agent-runtime pass-through by @yuneng-berri in #37386
fix(mcp): attach per-user BYOK credential when listing tools for non-oauth2 auth types by @devin-ai-integration[bot] in #34787
feat(ui): add success, warning and info status tokens by @yuneng-berri in #37393
refactor(ui): drop @tremor/react and the theming scaffolding it needed by @ryan-crabbe-berri in #37394
refactor(ui): move the model info edit form off antd Form by @yuneng-berri in #37392
fix(vector_stores): stop leaking stored credentials in direct search debug logs by @ryan-crabbe-berri in #37373
fix(logging): close three secret-leak paths in verbose logging by @ryan-crabbe-berri in #37391
chore: bump litellm-enterprise 0.1.56 -> 0.1.57, litellm-proxy-extras 0.4.86 -> 0.4.87, litellm 1.98.0 -> 1.99.0 by @yuneng-berri in #37395
fix(mcp): bind tool existence check to the selected server by @mateo-berri in #37388
fix(mcp): serve token-forwarding servers when oauth discovery fails by @tin-berri in #37399
fix(databricks): add cost map entries for 14 newer Databricks models by @epistoteles in #28501
refactor(ui): style the logging settings from semantic tokens by @yuneng-berri in #37385
feat(tinyfish): surface response headers + top-level response extras by @ChenluJi in #32448
refactor(ui): codemod the antd Tooltips outside form files onto the shadcn atom by @ryan-crabbe-berri in #37402
test(ocr): update Azure DI supported-params assertion for req_format by @mateo-berri in #37419
fix(proxy): return no rows when the aggregated activity entity filter is empty by @yuneng-berri in #37414
test: build redaction and batch limiter fixtures the way production does by @yuneng-berri in #37416
test: allow protocol-constrained pass-through routes to declare fewer methods by @yuneng-berri in #37415
test(ui): pin the MCP server edit save payload before the form migration by @yuneng-berri in #37404
test(ui): characterize the create key form payload contract by @yuneng-berri in #37405
refactor(ui): migrate the key edit form off Ant Design onto react-hook-form by @yuneng-berri in #37398
test(ui): repoint the e2e locators at the post-antd form controls by @yuneng-berri in #37421
test: point the live gemini and groq conformance suites at models that still exist by @yuneng-berri in #37422
refactor(ui): extract the create-key payload builder out of create_key_button by @yuneng-berri in #37397
fix(types): map nested prompt_tokens_details.cache_creation_input_tokens to cache_write_tokens by @devin-ai-integration[bot] in #37377
fix(router): honor key-level tag filtering in pre-routing and pin auto-router e2e regressions by @mateo-berri in #37366
feat(otel): attribute Prisma database spans to PostgreSQL instead of localhost by @yucheng-berri in #36595
fix(bedrock): degrade gracefully on malformed tool-call arguments by @devin-ai-integration[bot] in #33842
test: move the remaining live groq call sites off the retired llama models by @yuneng-berri in #37426
fix(ui): highlight the first member search match so Enter picks it by @yuneng-berri in #37429
chore(ci): promote internal staging to main by @yuneng-berri in #37400
fix(proxy): log spend for OpenAI passthrough embeddings with unmapped models by @mateo-berri in #37425
fix(router): keep acreate_file fallbacks inside the requested model group by @mateo-berri in #37424
fix(proxy): record estimated input tokens in spend logs for failed dispatched requests by @mateo-berri in #37365
fix: accept bool thinking param instead of crashing with AttributeError by @mateo-berri in #37423
refactor(ui): migrate the teams form graph off antd Form onto react-hook-form by @yuneng-berri in #37417
fix(ui): restore the cache control Role and Index field hints by @yuneng-berri in #37437
feat(ui): add mounted-field projections for the MCP server form graph by @yuneng-berri in #37440
refactor(ui): extract the MCP server edit save payload into a pure builder by @yuneng-berri in #37436
test: derive vertex batch cost expectation from the cost map by @mateo-berri in #37444
refactor(ui): port the create key form off antd Form onto react-hook-form by @yuneng-berri in #37442
refactor(ui): port the add model form off antd Form onto react-hook-form by @yuneng-berri in #37446
refactor(ui): host KeyLifecycleSettings tests in react-hook-form instead of antd Form by @yuneng-berri in #37449
fix(ui): rebuild nested and list paths in the mounted-field projection by @yuneng-berri in #37450
fix(ui): gate the pass-through guardrail field inputs when the section is disabled by @yuneng-berri in #37435
fix(tests): keep a host PROXY_BASE_URL out of request-derived URL tests by @mateo-berri in #37451
refactor(ui): port the MCP server forms off antd Form onto react-hook-form by @yuneng-berri in #37483
test(e2e): pin the tag-routing denial to its actual cause by @mateo-berri in #37432
fix(proxy): read through to the DB on registry misses so just-created models, guardrails, and agents resolve on sibling replicas by @mateo-berri in #36263
fix(mcp): forward the per-server auth header on OpenAPI tool calls by @tin-berri in #37410
chore(typing): drop 1.3k basedpyright errors across 42 Any hotspot files by @mateo-berri in #37439
test(ui): drive fields with change events where the typing is not the behaviour by @yuneng-berri in #37495
fix(ui): restore tab strip styling and panel persistence lost in the shadcn migration by @ryan-crabbe-berri in #37403
feat(spend-logs): add lifecycle timestamps by @sytianhe in #37361
refactor(ui): migrate the antd Button call sites onto the shadcn Button by @yuneng-berri in #37505
test(ui): split the vitest suite into unit, component, integration and type projects by @yuneng-berri in #37488
refactor(ptu): give the rollup a source-agnostic deployment record by @yucheng-berri in #37501
feat(auto-router)!: scope shadow eval jobs to multiple keys by @tin-berri in #37251
refactor(ui): migrate the antd Alert call sites onto the shared Alert by @yuneng-berri in #37513
chore(ui): upgrade the dashboard to React 19 by @ryan-crabbe-berri in #37411
fix(streaming): accept provider cost objects when propagating usage cost by @devin-ai-integration[bot] in #36593
fix(complexity-router): gate the reasoning override on a non-SIMPLE score by @tin-berri in #37500
fix(mcp): stop reporting failed OpenAPI tool calls as successes by @tin-berri in #37496
feat(e2e): add record/replay transport seam and fixture bundle format by @mateo-berri in #37360
fix(proxy): accept inherited model sentinels in project key limits by @mateo-berri in #37515
fix(model_prices): add provider-announced deprecation_date to 205 registry entries by @devin-ai-integration[bot] in #37283
fix(model_prices): correct gemini 3.1 flash image and deepseek v4 pricing, add openai deprecation dates by @devin-ai-integration[bot] in #37473
fix(model_prices): set prompt_cache_min_tokens=4096 for Gemini 3.5/3.6/3.7 Flash and 3.1 Pro Preview by @mateo-berri in #37516
fix(anthropic,bedrock): report provider thinking tokens instead of classifying them as text by @devin-ai-integration[bot] in #35998
fix(batches): stop one bad output line from zeroing an entire batch's spend by @mateo-berri in #37457
feat(e2e): canonical content-based match keys for record-and-replay by @mateo-berri in #37525
feat(cli): add lite login --config-claude to wire Claude Code at login by @yassin-berriai in #37507
fix(auth): resolve bare model names against wildcard deployments in model access groups by @ryan-crabbe-berri in #37492
docs: run only the tests covering your change, leave suites to CI by @devin-ai-integration[bot] in #37528
feat(complexity-router): make the reasoning override floor configurable by @tin-berri in #37537
fix(ui): drop stale user search answers so Enter commits the current match by @mateo-berri in #37504
refactor(ui): migrate the remaining dashboard pages off antd by @ryan-crabbe-berri in #37524
feat(proxy): auto-suppress the no-Redis banner for confirmed single-worker deployments by @mateo-berri in #36987
fix(proxy): retry spend updates on Postgres deadlock instead of dropping them by @RayJueWang in #34887
feat(search): add Amazon Bedrock AgentCore web search provider by @devin-ai-integration[bot] in #36331
fix(helm): default litellm-helm to the ghcr.io/berriai/litellm image by @yassin-berriai in #37491
refactor(ui): migrate antd Modal onto the shared shadcn Dialog by @yuneng-berri in #37540
fix(ui): toggle unlimited budget when its text is clicked by @yuneng-berri in #37547
feat(proxy): fast-fail validation for batch input files at /v1/files by @mateo-berri in #37527
chore: gitignore CLAUDE.local.md by @devin-ai-integration[bot] in #37545
perf(otel): build the credential-scoped tracer Resource once per logger by @yucheng-berri in #37542
fix(ci): gate backend unit tests on the pull request's own file list by @yuneng-berri in #37550
chore(codeowners): require pricing owner approval for the model prices jsons by @devin-ai-integration[bot] in #37551
fix(proxy): initialize the secret manager before resolving os.environ config references by @yassin-berriai in #37544
refactor(ui): swap @ant-design/icons for lucide-react by @ryan-crabbe-berri in #37553
refactor(ui): migrate shared primitives and common components off antd by @ryan-crabbe-berri in #37521
fix(spend-logs): backfill created_at/updated_at from row endTime instead of migration time by @mateo-berri in #37554
refactor(ui): migrate the MCP servers pages off antd by @ryan-crabbe-berri in #37522
fix(vertex_ai): apply regional endpoint uplift to cost tracking by @mateo-berri in #37543
fix(proxy): populate deployment attribution on failed-request spend logs by @mateo-berri in #37520
refactor(ui): migrate the model and router settings pages off antd by @ryan-crabbe-berri in #37523
perf(ci): gate the lint, MCP and dashboard jobs on the pull request's file list by @yuneng-berri in #37559
feat(router): allow per-tier litellm_params in complexity autorouter config by @devin-ai-integration[bot] in #37064
fix(anthropic): log partial stream spend when a /v1/messages client disconnects mid-stream by @mateo-berri in #37558
fix(ui): clear pass-through header rows when the create modal is reopened by @yuneng-berri in #37549
fix(ui): render optional array and object MCP tool parameters as JSON inputs by @yuneng-berri in #37548
feat(proxy)!: default audit logs on for enterprise licenses by @devin-ai-integration[bot] in #37518
feat(ui): standardize the Teams page header by @yuneng-berri in #36897
feat(ptu): accrue flat cost for PTU deployments declared in config.yaml by @yucheng-berri in #37556
refactor(ui): migrate the last antd components off antd onto shadcn by @ryan-crabbe-berri in #37569
chore(ui): drop the antd dependency and its leftovers by @ryan-crabbe-berri in #37574
refactor(ui): map hardcoded Tailwind palette classes onto semantic tokens by @ryan-crabbe-berri in #37576
fix(ptu): hand the prune a plain delete filter the query builder can serialise by @yucheng-berri in #37571
feat(proxy): enqueued-token rate limiting for batches with refund on completion and cancellation by @mateo-berri in #37539
fix(ui): restore hover feedback and dark-mode variants lost in the token migration by @yuneng-berri in #37579
fix(ci): run the full dashboard suite when a change reaches outside src/ by @yuneng-berri in #37563
fix(ui): keep semantic button colours on hover after the no-op hover cleanup by @yuneng-berri in #37580
fix: add supports_mid_conversation_system to bare first-party Claude cost-map keys by @oneKn8 in #36969
feat: add bedrock grok 4.6 to model cost map by @devin-ai-integration[bot] in #37517
fix: preserve prompt cache for mid-conversation system on unflagged Claude models by @oneKn8 in #36968
fix(router): routed deployment's own litellm_params beat forwarded auto_router marker params by @mateo-berri in #37615
refactor(ci): fold the nine thin unit-shard callers into one matrix by @yuneng-berri in #37590
chore(ci): close the test-census blind spots and move scripts out of workflows/ by @yuneng-berri in #37586
test: retire tests/old_proxy_tests, which holds no tests by @yuneng-berri in #37605
feat(ci): ratchet the test suite's zero-assert, mock-echo and global-state debt by @yuneng-berri in #37588
feat(proxy): native CLI login with OAuth authorization code + PKCE by @mateo-berri in #37626
feat(prompt-caching): map cache_control_injection_points to OpenAI prompt_cache_breakpoint on GPT-5.6+ targets by @mateo-berri in #37628
fix(realtime): bound Vertex credential resolution and make realtime failures loud by @mateo-berri in #37604
fix(anthropic): map metadata.user_id to prompt_cache_key on the /v1/messages bridge by @mateo-berri in #37623
fix(passthrough): resolve vertex live credentials from db model deployments by @mateo-berri in #37602
fix(prompt_management): don't route no-prompt_id requests to prompt managers that can't run them by @mateo-berri in #37575
test: remove the five test functions a later definition shadows by @yuneng-berri in #37591
feat(ci): guard shard assignment across every sharded test tree by @yuneng-berri in #37593
feat(ui): multi-key shadow eval picker and per-key breakdown by @tin-berri in #37389
fix(ui): make dark-mode form controls visible by @yuneng-berri in #37648
fix(ui): give status colours a readable foreground and drop the muted 70% step by @yuneng-berri in #37649
fix(ui): make inline styles and code blocks follow the theme by @yuneng-berri in #37651
fix(ui): move the policy flow builder onto theme tokens by @yuneng-berri in #37654
test: settle three allowlist entries that were open questions by @yuneng-berri in #37598
feat(ci): ratchet tests that skip themselves when a credential is absent by @yuneng-berri in #37612
feat(ci): catch files a -k expression deselects from every job by @yuneng-berri in #37601
test: run the 30 test files stranded in the second mirror by @yuneng-berri in #37595
fix(ui): make hardcoded palette surfaces theme-aware by @yuneng-berri in #37650
feat(cli): store the lite login credential in the OS keychain by @mateo-berri in #37566
fix(ui): draw one Per Day savings bar per date on Cost Optimization by @tin-berri in #37643
feat(mistral): add zai-glm-5-2 and glm-5-2 model pricing by @mubashir1osmani in #37110
feat(complexity_router): add business classification rubric preset by @devin-ai-integration[bot] in #37534
feat(ui): serve a dark-mode variant of the LiteLLM logo by @yuneng-berri in #37656
fix(otel): route Phoenix traces to per-key/team projects under otel v2 by @mubashir1osmani in #36706
test: replace blind sleeps with deadline waits in callback and caching tests by @ryan-crabbe-berri in #37660
fix(cli): keep the --pkce refresh token in the OS keychain, not in token.json by @mateo-berri in #37665
fix(ui): keep keyword tier rules that target operator-defined tiers when hydrating the edit modal by @tin-berri in #37413
feat(proxy): add POST /auto_router/validate_complexity_router_config to dry-run the complexity-router write gate by @tin-berri in #37409
feat(ui): let admins supply a dark-mode variant of their custom logo by @yuneng-berri in #37662
fix(proxy): run pre-call guardrails on batch input file uploads by @yucheng-berri in #37519
feat(ui): add a light/dark/system theme toggle to the top bar by @ryan-crabbe-berri in #37669
feat(proxy): redact or drop individual batch records instead of rejecting the file by @yucheng-berri in #37561
refactor(ui): mark dark as beta in the theme menu instead of the toolbar by @ryan-crabbe-berri in #37680
ci: lint the test tree for undefined names (F821) and fix all 30 by @ryan-crabbe-berri in #37671
feat(e2e): move record/replay to the provider edge (LIT-5745) by @mateo-berri in #37565
fix(mcp): let a salt-key-orphaned OAuth credential be replaced by re-authorization by @yassin-berriai in #37672
fix(mcp): normalize auth schemes so MCP egress emits exactly one prefix by @yassin-berriai in #37668
test: add six ruff rules that catch tests which cannot fail by @ryan-crabbe-berri in #37709
perf(ci): measure unit-shard coverage with the sys.monitoring core by @yuneng-berri in #37589
test: merge three stranded twins into the files that shadow them by @yuneng-berri in #37600
test(ci): reject coverage-allowlist entries that no longer match a file by @yuneng-berri in #37608
feat(ci): assert .github/workflows holds only workflows, correctly named by @yuneng-berri in #37616
feat(ci): freeze the conftest save/restore inventory so it can only shrink by @yuneng-berri in #37621
fix(a2a): accept the whole JSON-RPC id union the spec defines by @yassin-berriai in #37704
fix(ptu): refuse an incomplete config.yaml reservation the way the endpoints do by @yucheng-berri in #37703
chore: bump litellm-enterprise 0.1.57 -> 0.1.58, litellm-proxy-extras 0.4.87 -> 0.4.88 by @yuneng-berri in #37717
feat(perplexity): add Agent API third-party models by @mubashir1osmani in #37112
fix(ui): surface the paginated fallback on Cost Optimization by @tin-berri in #37659
feat(shadow_eval)!: gate the per-key budget on dollar spend instead of turns by @tin-berri in #37555
feat(ui): per-model reasoning effort in the complexity tier editor by @tin-berri in #37673
feat(proxy): authenticate to Azure Postgres with Microsoft Entra ID tokens by @mateo-berri in #37663
fix(scim): keep the matched user_id on POST /Users email match by @devin-ai-integration[bot] in #37701
fix(scim): fail group sync when a member add or user creation fails (LIT-5105) by @devin-ai-integration[bot] in #37688
fix(auth): cache team member default budget as a typed model (LIT-5311) by @devin-ai-integration[bot] in #37695
feat(proxy): add maximum_health_check_retention_period to bound the health-check table by @devin-ai-integration[bot] in #37681
fix(reset_budget_job): reconnect and retry on transient DB transport errors by @devin-ai-integration[bot] in #37705
fix(proxy): split agent inference and management routes so admin nodes can create agents by @ryan-crabbe-berri in #37730
feat(helm): compose DATABASE_URL_READ_REPLICA from a reader host secret key by @devin-ai-integration[bot] in #37109
fix(datadog): normalize alias-derived tag values so metrics match what was sent by @devin-ai-integration[bot] in #37682
fix(router): isolate deployment model info from cached backend metadata by @devin-ai-integration[bot] in #37687
fix(logging): bound the shared logging executor backlog by @devin-ai-integration[bot] in #37694
fix(proxy): capture requester IP in 401 and auth-time 429 failure logs by @devin-ai-integration[bot] in #37707
fix(logging): bound oversized error payloads written to stdout by @devin-ai-integration[bot] in #37684
perf(budget_reservation): tokenize each request once, off the event loop for large prompts by @devin-ai-integration[bot] in #37683
perf(prometheus): render /metrics off the event loop and coalesce concurrent scrapes by @devin-ai-integration[bot] in #37702
fix(token_counter): stop large token counts from blocking the proxy event loop by @devin-ai-integration[bot] in #37697
fix(router): give auto prompt caching deployment affinity (LIT-5535) by @devin-ai-integration[bot] in #37689
test(e2e): pin prompt-cache, service-tier, and cost-header billing as permanent regressions by @mateo-berri in #37607
feat(rust): route /chat/completions through the Rust core for anthropic and bedrock by @yassin-berriai in #37241
test(e2e): pin query params and multipart form fields as replay match-key identity by @mubashir1osmani in #37679
fix(a2a): return SSE (text/event-stream) for message/stream instead of NDJSON by @devin-ai-integration[bot] in #35037
fix(proxy): fail the standalone prisma migration entrypoint on migration errors by @devin-ai-integration[bot] in #37692
feat(sso): source generic OIDC user claims from ID/access token when UserInfo is incomplete by @devin-ai-integration[bot] in #37696
fix(proxy): claim batch cost rows atomically so multi-pod polling can't double-bill by @devin-ai-integration[bot] in #37685
fix(jwt): retry JWKS fetches, serve stale keys, and return 503 when the IdP is unreachable by @devin-ai-integration[bot] in #37690
fix(ui): stop the Add Model mapping table from looping the page by @yuneng-berri in #37741
fix(containers): surface provider errors from container file content endpoint by @yuneng-berri in #37737
test(ci): serve /moderations from the canned OpenAI mock by @yuneng-berri in #37739
test: point the live web search, groq and vertex image suites at models that still exist by @yuneng-berri in #37733
fix(model-costs): correct gpt-5.6 max input tokens to 922k by @devin-ai-integration[bot] in #37722
fix(db): apply the configured connection params to the read replica URL by @devin-ai-integration[bot] in #37691
feat(fal_ai): add gpt-image-2 image generation support by @devin-ai-integration[bot] in #37729
fix(scim): propagate team roster write failures on group and user writes by @devin-ai-integration[bot] in #37700
fix(redis): apply Azure AD and GCP IAM auth to every async client path by @mateo-berri in #37740
test: assert the prefixed model the responses bridge now hands back by @yuneng-berri in #37744
perf(reset_budget_job): elect one sweeper per tick and bound the window scan by @yassin-berriai in #36497
feat(proxy): add admin toggle to block requests for models without pricing by @devin-ai-integration[bot] in #35181
fix(model_prices): consolidate eleven open registry audits into one changeset by @devin-ai-integration[bot] in #37658
feat(ui): edit project input and output TPM limits from the Projects modal by @devin-ai-integration[bot] in #37676
fix(vertex_ai): only fall back to a placeholder thought signature on the first parallel function call by @ljogeiger in #37541
test(lint): ban blind pytest.raises(Exception) with ruff B017 by @ryan-crabbe-berri in #37731
fix(responses_bridge): map incomplete responses to finish_reason length instead of 500 by @mateo-berri in #37710
feat(cognition): give Cognition its own provider identity by @mateo-berri in #37743
fix: add moonshot/kimi-k3 to the cost map on main by @mateo-berri in #37753
fix(caching): bound the semantic cache embedding lookup so a dead embedding endpoint can't block requests by @mateo-berri in #37742
test: assert the prefixed model the azure responses bridge now hands back by @yuneng-berri in #37749
test: point the live together_ai suites at a model together still serves by @yuneng-berri in #37746
test(e2e): pin openai_passthrough routing, cost logging, and file list isolation by @mateo-berri in #37618
chore(ci): promote internal staging to main by @yuneng-berri in #37721
fix(scim): match group members by SSO identity or email before creating a placeholder by @devin-ai-integration[bot] in #37686
fix(cost): match streamed Messages usage cost to the recorded spend by @devin-ai-integration[bot] in #35114
test: enforce PT012 so a pytest.raises block cannot hide dead assertions by @ryan-crabbe-berri in #37748
fix(streaming): price partial-stream spend rows at the real model and keep prompt and cache fields by @mateo-berri in #37734
fix(fal_ai): price gpt-image-2 per size and quality from request params by @mateo-berri in #37751
test: enforce PT011 and PT014 so a broad pytest.raises cannot pass on the wrong error by @ryan-crabbe-berri in #37769
fix: populate team member emails missing from the roster snapshot by @ryan-crabbe-berri in #37759
fix(bedrock): forward provider response headers on chat completions by @devin-ai-integration[bot] in #37003
fix(cognition): price swe-1.7 at the standard tier, add swe-1.7-lightning by @mateo-berri in #37763
fix(sagemaker_chat): send the inference component header and honor hf_model_name by @mateo-berri in #37766
fix(spend-tracking): hash raw api keys before persisting to spend logs by @nitishagar in #30736
fix(proxy): make per-model budgets track spend, enforce, and report the same counter by @yassin-berriai in #37736
fix(spend): bound each spend-log write statement by row count as well as bytes by @yassin-berriai in #37758
feat(proxy): add router_model_name to auto-routed response bodies by @tin-berri in #37725
fix(responses): preserve Bedrock Mantle validation errors by @uuzzrm in #36580
fix: omit thinking.type=disabled for always-on thinking Claude models by @tin-berri in #37510
test(lint): clear the two PT011/PT012 violations left on the test tree by @tin-berri in #37864
fix(proxy): read batch records the same way the upload validation does by @yucheng-berri in #37776
fix(proxy): scan batch records with the content hooks that are not guardrails by @yucheng-berri in #37786
fix(ptu): require an operator-declared id on a config.yaml reservation by @yucheng-berri in #37794
fix(ptu): never retract a flat charge for a deployment the run cannot see by @yucheng-berri in #37793
chore(codeowners): own the proxy-extras migrations directory by @yuneng-berri in #37876
revert(spend-logs): drop the endTime backfill migration for spend log timestamps by @yuneng-berri in #37875
fix(mcp): deny the interactive dcr_bridge authorize for a user without server access by @tin-berri in #37865
fix(responses): map Bedrock Mantle context overflow to ContextWindowExceededError by @yassin-berriai in #37862
test: enforce F811 so a duplicate definition cannot silently replace the first by @ryan-crabbe-berri in #37878
fix(guardrails): run policy pipelines when the caller sends its own metadata (/v1/messages, Claude Code) by @devin-ai-integration[bot] in #36889
fix(vertex_ai): convert messages to contents in gemini count_tokens by @sailikhithk in #36981
fix(router): don't log 'Could not identify azure model' when the deployment name resolves from the cost map by @tin-berri in #37869
fix(ui): drive auto-router usage from the shared cost-optimization time picker by @tin-berri in #37871
feat(ui): add per-key Savings tab to key detail page by @tin-berri in #37693
fix(redis): reset only the failed node on a cluster client timeout, not the whole client by @yassin-berriai in #37863
test: reject assertions on a caught error inside except (ruff PT017) by @ryan-crabbe-berri in #37887
fix(mcp): strip root_path before matching the per-server MCP route spelling by @tin-berri in #35576
feat(spend): surface per-request auto-router savings to logging callbacks by @tin-berri in #37894
fix(proxy): group Codex turns under one session id by @mubashir1osmani in #37895
fix(logging): preserve uvicorn color_message args during secret redaction by @mubashir1osmani in #37122
feat(llm): add moonshot/kimi-k3 to model prices and context window map by @tin-berri in #37552
feat(providers): add SCX.ai as a JSON-configured OpenAI-compatible provider by @bhuvan2134686 in #34752
feat(bedrock): serve gpt-5.6 cross-region inference profiles on bedrock runtime by @longwind48 in #37821
fix(batches): decode model-encoded output file id so completed batches book spend by @mateo-berri in #37573
fix(model_prices): correct gemini-3.1-flash-lite-image capabilities and dedupe its entries by @emerzon in #36849
test: say whether a match= pattern is a regex or a literal (ruff RUF043) by @ryan-crabbe-berri in #37901
fix(mcp): resolve admin OAuth sessions to the same server set the connect page shows by @tin-berri in #37900
fix(anthropic_messages): gate sampling params on /v1/messages like /chat/completions by @tin-berri in #37868
test(e2e): pin the openai websocket passthrough prefixes by @mateo-berri in #37903
fix(model-costs): apply GPT-5.6 Sol promotional pricing cut by @devin-ai-integration[bot] in #37880
fix(pricing): add undated azure aliases for gpt-audio-mini and gpt-realtime-mini by @tin-berri in #37867
fix(anthropic): resolve the provider exactly once on /v1/messages by @tin-berri in #37757
test(e2e): record and replay the non-streaming provider flows by @mateo-berri in #37909
test(mcp): retire the last file of the dead tests/litellm mirror by @yuneng-berri in #37824
perf(ci): cache the Rust build the unit shards compile from scratch by @yuneng-berri in #37795
test(proxy): pin the request-body rules proxy/_types.py enforces by @yuneng-berri in #37811
test(main): pin what a streamed response costs, end to end by @yuneng-berri in #37812
test(proxy): pin what a failed request records as usage and spend by @yuneng-berri in #37813
test(llm_http_handler): pin the websocket and callback gates the request path branches on by @yuneng-berri in #37814
fix(ci): stop the mutation report publishing a score it never measured by @yuneng-berri in #37825
test(bedrock): let monkeypatch own bedrock_request_metadata_fields by @yuneng-berri in #37840
test(audit-logs): let monkeypatch own the audit log and s3 callback globals by @yuneng-berri in #37842
test: unwind environment writes in tests/test_litellm with monkeypatch by @yuneng-berri in #37806
test(datadog): restore an empty DD_API_KEY instead of unsetting it by @yuneng-berri in #37832
test(cost-calc): stop 182 global writes leaking out of the cost-calc suites by @yuneng-berri in #37815
test(key-management): unwind the global writes the key tests scaffold around by @yuneng-berri in #37822
test(callbacks): unwind the callbacks global the policy engine and realtime tests scaffold around by @yuneng-berri in #37826
test(guardrails): stop the onyx and prompt security tests leaking env vars on failure by @yuneng-berri in #37828
test(e2e): bypass the proxy response cache in the mid-conversation system and fallback tests by @yuneng-berri in #37915
docs(terraform/provider): the provider now ships at the LiteLLM version by @yuneng-berri in #37912
test: stop the zai tests from leaking env and litellm globals into the session by @yuneng-berri in #37834
test(s3): stop the logger tests leaking s3_callback_params on failure by @yuneng-berri in #37831
test(http-handler): drop the save/restore scaffolding around litellm globals by @yuneng-berri in #37839
test(interactions): drop the save/restore scaffolding around the legacy flag by @yuneng-berri in #37841
test(bedrock): drop the leftover set_verbose from the embedding tests by @yuneng-berri in #37844
test: drop the leftover set_verbose from eleven test files by @yuneng-berri in #37845
fix(ptu): warn when config.yaml declares PTU while attribution is off by @yucheng-berri in #37898
test(anthropic): let monkeypatch own litellm.callbacks in the cache control tests by @yuneng-berri in #37847
test(router): let monkeypatch own expose_router_debug_in_errors by @yuneng-berri in #37848
fix(proxy): stop forwarding a client Anthropic OAuth token to Bedrock and Vertex by @mateo-berri in #37905
chore: bump litellm-enterprise 0.1.58 -> 0.1.59, litellm-proxy-extras 0.4.88 -> 0.4.89 by @yuneng-berri in #37939
test: unshadow the module handles the F811 sweep left behind by @yuneng-berri in #37914
test: drop the cwd-relative sys.path.insert calls from the test suite by @yuneng-berri in #37802
chore(codeowners): add yuneng-berri as owner of the CODEOWNERS file by @yuneng-berri in #37944
fix(files): list and page unscoped managed files locally by @rimysore in #37855
fix(otel): emit LLM Call spans for speech, image, moderation, ocr and transcription by @devin-ai-integration[bot] in #37752
fix(ui): keep completion-mode models in the playground chat dropdown by @yuneng-berri in #37954
fix(types): silence pydantic ReadOnly warning on StandardLoggingRoutingDecision by @mateo-berri in #37949
fix(websearch_interception): end the turn when the agentic loop hits its ceiling by @mateo-berri in #37911
fix(proxy): keep a failed prisma generate from failing the migration entrypoint by @yuneng-berri in #37947
fix(docker): bump wolfi-base digest for busybox 1.38.0-r1 and openssl 3.6.3-r5 by @yuneng-berri in #37950
fix: don't retire a completed batch from cost recovery while output_file_id is lagging by @syahra712 in #37715
fix(guardrails): pii/pci guardrail masking gaps in SpendLogs, debug logs, and logging_only response by @yassin-berriai in #37965
fix(proxy): make /team/member_delete's four cleanups atomic by @yassin-berriai in #37959
fix(auth): resolve team object_permission independently in the unresolvable-team fallback by @yassin-berriai in #37960
fix(auth): stop the team fallback from widening model access by @yassin-berriai in #37962
fix(prometheus): fold auth/pre-call time into litellm_request_total_latency_metric by @yassin-berriai in #37958
fix(responses): mint Responses API item IDs in the completion bridge by @mateo-berri in #37946
test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips by @yuneng-berri in #37957
fix(proxy): omit litellm_batch_guardrail when no guardrail acted by @yucheng-berri in #37964
fix(responses-bridge): preserve reasoning input items and signed thinking blocks by @harryzhou2000 in #36355
chore: rebuild Admin UI bundle by @yuneng-berri in #37976
test: add regression coverage for twelve closed issues by @mateo-berri in #37974
chore(ci): promote internal staging to main by @yuneng-berri in #37913
fix(ui): backport the shadcn migration regression fixes onto 1.99.0 by @yuneng-berri in #38821
fix(e2e): backport the vertex realtime and vision image fixture fixes to rc/1.99.0 by @yuneng-berri in #38848
fix(ui): drop stray text next to Close in the model connection test dialog by @yuneng-berri in #38854
[Backport rc/1.99.0] fix(anthropic): translate tool_result document blocks in the /v1/messages bridge by @yuneng-berri in #38855
test(e2e): backport the select-anchoring and router-fallback spec de-flakes by @yuneng-berri in #38862
fix(docker): pin apk python to 3.13 on rc/1.99.0 (cherry-pick #38917) by @yuneng-berri in #39048

New Contributors

@LHMQ878 made their first contribution in #36151
@ArjunPakhan made their first contribution in #34087
@MUSE-CODE-SPACE made their first contribution in #34067
@brian5021 made their first contribution in #36878
@Scott-Wilson-ZocDoc made their first contribution in #36978
@irosh-colombage-ZocDoc2 made their first contribution in #36482
@itaimodi made their first contribution in #36764
@ayaangazali made their first contribution in #34445
@bruno-olivia made their first contribution in #37242
@Sujithr07 made their first contribution in #33195
@shivijain2323 made their first contribution in #35110
@ChenluJi made their first contribution in #32448
@sytianhe made their first contribution in #37361
@RayJueWang made their first contribution in #34887
@oneKn8 made their first contribution in #36969
@uuzzrm made their first contribution in #36580
@sailikhithk made their first contribution in #36981
@bhuvan2134686 made their first contribution in #34752
@longwind48 made their first contribution in #37821
@syahra712 made their first contribution in #37715
@harryzhou2000 made their first contribution in #36355

Full Changelog: v1.98.0...v1.99.0