BerriAI/litellm v1.99.0-dev.2 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.99.0-dev.2

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.99.0-dev.2/cosign.pub \
  ghcr.io/berriai/litellm:v1.99.0-dev.2

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

fix(proxy): log spend for OpenAI passthrough embeddings with unmapped models by @mateo-berri in #37425
fix(router): keep acreate_file fallbacks inside the requested model group by @mateo-berri in #37424
fix(proxy): record estimated input tokens in spend logs for failed dispatched requests by @mateo-berri in #37365
fix: accept bool thinking param instead of crashing with AttributeError by @mateo-berri in #37423
refactor(ui): migrate the teams form graph off antd Form onto react-hook-form by @yuneng-berri in #37417
fix(ui): restore the cache control Role and Index field hints by @yuneng-berri in #37437
feat(ui): add mounted-field projections for the MCP server form graph by @yuneng-berri in #37440
refactor(ui): extract the MCP server edit save payload into a pure builder by @yuneng-berri in #37436
test: derive vertex batch cost expectation from the cost map by @mateo-berri in #37444
refactor(ui): port the create key form off antd Form onto react-hook-form by @yuneng-berri in #37442
refactor(ui): port the add model form off antd Form onto react-hook-form by @yuneng-berri in #37446
refactor(ui): host KeyLifecycleSettings tests in react-hook-form instead of antd Form by @yuneng-berri in #37449
fix(ui): rebuild nested and list paths in the mounted-field projection by @yuneng-berri in #37450
fix(ui): gate the pass-through guardrail field inputs when the section is disabled by @yuneng-berri in #37435
fix(tests): keep a host PROXY_BASE_URL out of request-derived URL tests by @mateo-berri in #37451
refactor(ui): port the MCP server forms off antd Form onto react-hook-form by @yuneng-berri in #37483
test(e2e): pin the tag-routing denial to its actual cause by @mateo-berri in #37432
fix(proxy): read through to the DB on registry misses so just-created models, guardrails, and agents resolve on sibling replicas by @mateo-berri in #36263
fix(mcp): forward the per-server auth header on OpenAPI tool calls by @tin-berri in #37410
chore(typing): drop 1.3k basedpyright errors across 42 Any hotspot files by @mateo-berri in #37439
test(ui): drive fields with change events where the typing is not the behaviour by @yuneng-berri in #37495
fix(ui): restore tab strip styling and panel persistence lost in the shadcn migration by @ryan-crabbe-berri in #37403
feat(spend-logs): add lifecycle timestamps by @sytianhe in #37361
refactor(ui): migrate the antd Button call sites onto the shadcn Button by @yuneng-berri in #37505
test(ui): split the vitest suite into unit, component, integration and type projects by @yuneng-berri in #37488
refactor(ptu): give the rollup a source-agnostic deployment record by @yucheng-berri in #37501
feat(auto-router)!: scope shadow eval jobs to multiple keys by @tin-berri in #37251
refactor(ui): migrate the antd Alert call sites onto the shared Alert by @yuneng-berri in #37513
chore(ui): upgrade the dashboard to React 19 by @ryan-crabbe-berri in #37411
fix(streaming): accept provider cost objects when propagating usage cost by @devin-ai-integration[bot] in #36593
fix(complexity-router): gate the reasoning override on a non-SIMPLE score by @tin-berri in #37500
fix(mcp): stop reporting failed OpenAPI tool calls as successes by @tin-berri in #37496
feat(e2e): add record/replay transport seam and fixture bundle format by @mateo-berri in #37360
fix(proxy): accept inherited model sentinels in project key limits by @mateo-berri in #37515
fix(model_prices): add provider-announced deprecation_date to 205 registry entries by @devin-ai-integration[bot] in #37283
fix(model_prices): correct gemini 3.1 flash image and deepseek v4 pricing, add openai deprecation dates by @devin-ai-integration[bot] in #37473
fix(model_prices): set prompt_cache_min_tokens=4096 for Gemini 3.5/3.6/3.7 Flash and 3.1 Pro Preview by @mateo-berri in #37516
fix(anthropic,bedrock): report provider thinking tokens instead of classifying them as text by @devin-ai-integration[bot] in #35998
fix(batches): stop one bad output line from zeroing an entire batch's spend by @mateo-berri in #37457
feat(e2e): canonical content-based match keys for record-and-replay by @mateo-berri in #37525
feat(cli): add lite login --config-claude to wire Claude Code at login by @yassin-berriai in #37507
fix(auth): resolve bare model names against wildcard deployments in model access groups by @ryan-crabbe-berri in #37492
docs: run only the tests covering your change, leave suites to CI by @devin-ai-integration[bot] in #37528
feat(complexity-router): make the reasoning override floor configurable by @tin-berri in #37537
fix(ui): drop stale user search answers so Enter commits the current match by @mateo-berri in #37504
refactor(ui): migrate the remaining dashboard pages off antd by @ryan-crabbe-berri in #37524
feat(proxy): auto-suppress the no-Redis banner for confirmed single-worker deployments by @mateo-berri in #36987
fix(proxy): retry spend updates on Postgres deadlock instead of dropping them by @RayJueWang in #34887
feat(search): add Amazon Bedrock AgentCore web search provider by @devin-ai-integration[bot] in #36331
fix(helm): default litellm-helm to the ghcr.io/berriai/litellm image by @yassin-berriai in #37491
refactor(ui): migrate antd Modal onto the shared shadcn Dialog by @yuneng-berri in #37540
fix(ui): toggle unlimited budget when its text is clicked by @yuneng-berri in #37547
feat(proxy): fast-fail validation for batch input files at /v1/files by @mateo-berri in #37527
chore: gitignore CLAUDE.local.md by @devin-ai-integration[bot] in #37545
perf(otel): build the credential-scoped tracer Resource once per logger by @yucheng-berri in #37542
fix(ci): gate backend unit tests on the pull request's own file list by @yuneng-berri in #37550
chore(codeowners): require pricing owner approval for the model prices jsons by @devin-ai-integration[bot] in #37551
fix(proxy): initialize the secret manager before resolving os.environ config references by @yassin-berriai in #37544
refactor(ui): swap @ant-design/icons for lucide-react by @ryan-crabbe-berri in #37553
refactor(ui): migrate shared primitives and common components off antd by @ryan-crabbe-berri in #37521
fix(spend-logs): backfill created_at/updated_at from row endTime instead of migration time by @mateo-berri in #37554
refactor(ui): migrate the MCP servers pages off antd by @ryan-crabbe-berri in #37522
fix(vertex_ai): apply regional endpoint uplift to cost tracking by @mateo-berri in #37543
fix(proxy): populate deployment attribution on failed-request spend logs by @mateo-berri in #37520
refactor(ui): migrate the model and router settings pages off antd by @ryan-crabbe-berri in #37523
perf(ci): gate the lint, MCP and dashboard jobs on the pull request's file list by @yuneng-berri in #37559
feat(router): allow per-tier litellm_params in complexity autorouter config by @devin-ai-integration[bot] in #37064
fix(anthropic): log partial stream spend when a /v1/messages client disconnects mid-stream by @mateo-berri in #37558
fix(ui): clear pass-through header rows when the create modal is reopened by @yuneng-berri in #37549
fix(ui): render optional array and object MCP tool parameters as JSON inputs by @yuneng-berri in #37548
feat(proxy)!: default audit logs on for enterprise licenses by @devin-ai-integration[bot] in #37518
feat(ui): standardize the Teams page header by @yuneng-berri in #36897
feat(ptu): accrue flat cost for PTU deployments declared in config.yaml by @yucheng-berri in #37556
refactor(ui): migrate the last antd components off antd onto shadcn by @ryan-crabbe-berri in #37569
chore(ui): drop the antd dependency and its leftovers by @ryan-crabbe-berri in #37574
refactor(ui): map hardcoded Tailwind palette classes onto semantic tokens by @ryan-crabbe-berri in #37576
fix(ptu): hand the prune a plain delete filter the query builder can serialise by @yucheng-berri in #37571
feat(proxy): enqueued-token rate limiting for batches with refund on completion and cancellation by @mateo-berri in #37539
fix(ui): restore hover feedback and dark-mode variants lost in the token migration by @yuneng-berri in #37579
fix(ci): run the full dashboard suite when a change reaches outside src/ by @yuneng-berri in #37563
fix(ui): keep semantic button colours on hover after the no-op hover cleanup by @yuneng-berri in #37580
fix: add supports_mid_conversation_system to bare first-party Claude cost-map keys by @oneKn8 in #36969
feat: add bedrock grok 4.6 to model cost map by @devin-ai-integration[bot] in #37517
fix: preserve prompt cache for mid-conversation system on unflagged Claude models by @oneKn8 in #36968
fix(router): routed deployment's own litellm_params beat forwarded auto_router marker params by @mateo-berri in #37615
refactor(ci): fold the nine thin unit-shard callers into one matrix by @yuneng-berri in #37590
chore(ci): close the test-census blind spots and move scripts out of workflows/ by @yuneng-berri in #37586
test: retire tests/old_proxy_tests, which holds no tests by @yuneng-berri in #37605
feat(ci): ratchet the test suite's zero-assert, mock-echo and global-state debt by @yuneng-berri in #37588
feat(proxy): native CLI login with OAuth authorization code + PKCE by @mateo-berri in #37626
feat(prompt-caching): map cache_control_injection_points to OpenAI prompt_cache_breakpoint on GPT-5.6+ targets by @mateo-berri in #37628
fix(realtime): bound Vertex credential resolution and make realtime failures loud by @mateo-berri in #37604
fix(anthropic): map metadata.user_id to prompt_cache_key on the /v1/messages bridge by @mateo-berri in #37623
fix(passthrough): resolve vertex live credentials from db model deployments by @mateo-berri in #37602
fix(prompt_management): don't route no-prompt_id requests to prompt managers that can't run them by @mateo-berri in #37575
test: remove the five test functions a later definition shadows by @yuneng-berri in #37591
feat(ci): guard shard assignment across every sharded test tree by @yuneng-berri in #37593
feat(ui): multi-key shadow eval picker and per-key breakdown by @tin-berri in #37389
fix(ui): make dark-mode form controls visible by @yuneng-berri in #37648
fix(ui): give status colours a readable foreground and drop the muted 70% step by @yuneng-berri in #37649
fix(ui): make inline styles and code blocks follow the theme by @yuneng-berri in #37651
fix(ui): move the policy flow builder onto theme tokens by @yuneng-berri in #37654
test: settle three allowlist entries that were open questions by @yuneng-berri in #37598
feat(ci): ratchet tests that skip themselves when a credential is absent by @yuneng-berri in #37612
feat(ci): catch files a -k expression deselects from every job by @yuneng-berri in #37601
test: run the 30 test files stranded in the second mirror by @yuneng-berri in #37595
fix(ui): make hardcoded palette surfaces theme-aware by @yuneng-berri in #37650
feat(cli): store the lite login credential in the OS keychain by @mateo-berri in #37566
fix(ui): draw one Per Day savings bar per date on Cost Optimization by @tin-berri in #37643
feat(mistral): add zai-glm-5-2 and glm-5-2 model pricing by @mubashir1osmani in #37110
feat(complexity_router): add business classification rubric preset by @devin-ai-integration[bot] in #37534
feat(ui): serve a dark-mode variant of the LiteLLM logo by @yuneng-berri in #37656
fix(otel): route Phoenix traces to per-key/team projects under otel v2 by @mubashir1osmani in #36706
test: replace blind sleeps with deadline waits in callback and caching tests by @ryan-crabbe-berri in #37660
fix(cli): keep the --pkce refresh token in the OS keychain, not in token.json by @mateo-berri in #37665
fix(ui): keep keyword tier rules that target operator-defined tiers when hydrating the edit modal by @tin-berri in #37413
feat(proxy): add POST /auto_router/validate_complexity_router_config to dry-run the complexity-router write gate by @tin-berri in #37409
feat(ui): let admins supply a dark-mode variant of their custom logo by @yuneng-berri in #37662
fix(proxy): run pre-call guardrails on batch input file uploads by @yucheng-berri in #37519
feat(ui): add a light/dark/system theme toggle to the top bar by @ryan-crabbe-berri in #37669
feat(proxy): redact or drop individual batch records instead of rejecting the file by @yucheng-berri in #37561
refactor(ui): mark dark as beta in the theme menu instead of the toolbar by @ryan-crabbe-berri in #37680
ci: lint the test tree for undefined names (F821) and fix all 30 by @ryan-crabbe-berri in #37671
feat(e2e): move record/replay to the provider edge (LIT-5745) by @mateo-berri in #37565
fix(mcp): let a salt-key-orphaned OAuth credential be replaced by re-authorization by @yassin-berriai in #37672
fix(mcp): normalize auth schemes so MCP egress emits exactly one prefix by @yassin-berriai in #37668
test: add six ruff rules that catch tests which cannot fail by @ryan-crabbe-berri in #37709
perf(ci): measure unit-shard coverage with the sys.monitoring core by @yuneng-berri in #37589
test: merge three stranded twins into the files that shadow them by @yuneng-berri in #37600
test(ci): reject coverage-allowlist entries that no longer match a file by @yuneng-berri in #37608
feat(ci): assert .github/workflows holds only workflows, correctly named by @yuneng-berri in #37616
feat(ci): freeze the conftest save/restore inventory so it can only shrink by @yuneng-berri in #37621
fix(a2a): accept the whole JSON-RPC id union the spec defines by @yassin-berriai in #37704
fix(ptu): refuse an incomplete config.yaml reservation the way the endpoints do by @yucheng-berri in #37703
chore: bump litellm-enterprise 0.1.57 -> 0.1.58, litellm-proxy-extras 0.4.87 -> 0.4.88 by @yuneng-berri in #37717
feat(perplexity): add Agent API third-party models by @mubashir1osmani in #37112
fix(ui): surface the paginated fallback on Cost Optimization by @tin-berri in #37659
feat(shadow_eval)!: gate the per-key budget on dollar spend instead of turns by @tin-berri in #37555
feat(ui): per-model reasoning effort in the complexity tier editor by @tin-berri in #37673
feat(proxy): authenticate to Azure Postgres with Microsoft Entra ID tokens by @mateo-berri in #37663
fix(scim): keep the matched user_id on POST /Users email match by @devin-ai-integration[bot] in #37701
fix(scim): fail group sync when a member add or user creation fails (LIT-5105) by @devin-ai-integration[bot] in #37688
fix(auth): cache team member default budget as a typed model (LIT-5311) by @devin-ai-integration[bot] in #37695
feat(proxy): add maximum_health_check_retention_period to bound the health-check table by @devin-ai-integration[bot] in #37681
fix(reset_budget_job): reconnect and retry on transient DB transport errors by @devin-ai-integration[bot] in #37705
fix(proxy): split agent inference and management routes so admin nodes can create agents by @ryan-crabbe-berri in #37730
feat(helm): compose DATABASE_URL_READ_REPLICA from a reader host secret key by @devin-ai-integration[bot] in #37109
fix(datadog): normalize alias-derived tag values so metrics match what was sent by @devin-ai-integration[bot] in #37682
fix(router): isolate deployment model info from cached backend metadata by @devin-ai-integration[bot] in #37687
fix(logging): bound the shared logging executor backlog by @devin-ai-integration[bot] in #37694
fix(proxy): capture requester IP in 401 and auth-time 429 failure logs by @devin-ai-integration[bot] in #37707
fix(logging): bound oversized error payloads written to stdout by @devin-ai-integration[bot] in #37684
perf(budget_reservation): tokenize each request once, off the event loop for large prompts by @devin-ai-integration[bot] in #37683
perf(prometheus): render /metrics off the event loop and coalesce concurrent scrapes by @devin-ai-integration[bot] in #37702
fix(token_counter): stop large token counts from blocking the proxy event loop by @devin-ai-integration[bot] in #37697
fix(router): give auto prompt caching deployment affinity (LIT-5535) by @devin-ai-integration[bot] in #37689
test(e2e): pin prompt-cache, service-tier, and cost-header billing as permanent regressions by @mateo-berri in #37607
feat(rust): route /chat/completions through the Rust core for anthropic and bedrock by @yassin-berriai in #37241
test(e2e): pin query params and multipart form fields as replay match-key identity by @mubashir1osmani in #37679
fix(a2a): return SSE (text/event-stream) for message/stream instead of NDJSON by @devin-ai-integration[bot] in #35037
fix(proxy): fail the standalone prisma migration entrypoint on migration errors by @devin-ai-integration[bot] in #37692
feat(sso): source generic OIDC user claims from ID/access token when UserInfo is incomplete by @devin-ai-integration[bot] in #37696
fix(proxy): claim batch cost rows atomically so multi-pod polling can't double-bill by @devin-ai-integration[bot] in #37685
fix(jwt): retry JWKS fetches, serve stale keys, and return 503 when the IdP is unreachable by @devin-ai-integration[bot] in #37690
fix(ui): stop the Add Model mapping table from looping the page by @yuneng-berri in #37741
fix(containers): surface provider errors from container file content endpoint by @yuneng-berri in #37737
test(ci): serve /moderations from the canned OpenAI mock by @yuneng-berri in #37739
test: point the live web search, groq and vertex image suites at models that still exist by @yuneng-berri in #37733
fix(model-costs): correct gpt-5.6 max input tokens to 922k by @devin-ai-integration[bot] in #37722
fix(db): apply the configured connection params to the read replica URL by @devin-ai-integration[bot] in #37691
feat(fal_ai): add gpt-image-2 image generation support by @devin-ai-integration[bot] in #37729
fix(scim): propagate team roster write failures on group and user writes by @devin-ai-integration[bot] in #37700
fix(redis): apply Azure AD and GCP IAM auth to every async client path by @mateo-berri in #37740
test: assert the prefixed model the responses bridge now hands back by @yuneng-berri in #37744
perf(reset_budget_job): elect one sweeper per tick and bound the window scan by @yassin-berriai in #36497
feat(proxy): add admin toggle to block requests for models without pricing by @devin-ai-integration[bot] in #35181
fix(model_prices): consolidate eleven open registry audits into one changeset by @devin-ai-integration[bot] in #37658
feat(ui): edit project input and output TPM limits from the Projects modal by @devin-ai-integration[bot] in #37676
fix(vertex_ai): only fall back to a placeholder thought signature on the first parallel function call by @ljogeiger in #37541
test(lint): ban blind pytest.raises(Exception) with ruff B017 by @ryan-crabbe-berri in #37731
fix(responses_bridge): map incomplete responses to finish_reason length instead of 500 by @mateo-berri in #37710
feat(cognition): give Cognition its own provider identity by @mateo-berri in #37743
fix: add moonshot/kimi-k3 to the cost map on main by @mateo-berri in #37753
fix(caching): bound the semantic cache embedding lookup so a dead embedding endpoint can't block requests by @mateo-berri in #37742
test: assert the prefixed model the azure responses bridge now hands back by @yuneng-berri in #37749
test: point the live together_ai suites at a model together still serves by @yuneng-berri in #37746
test(e2e): pin openai_passthrough routing, cost logging, and file list isolation by @mateo-berri in #37618
chore(ci): promote internal staging to main by @yuneng-berri in #37721

New Contributors

@sytianhe made their first contribution in #37361
@RayJueWang made their first contribution in #34887
@oneKn8 made their first contribution in #36969

Full Changelog: v1.99.0-dev.1...v1.99.0-dev.2