BerriAI/litellm v1.97.0-dev.2 on GitHub

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. Every release is signed with the same key introduced in commit 0112e53.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.97.0-dev.2

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.97.0-dev.2/cosign.pub \
  ghcr.io/berriai/litellm:v1.97.0-dev.2

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

feat(ui): add role capability gating, migrate Tool Policies route by @yuneng-berri in #35812
refactor(ui): inject the fetch client's base url instead of reading it at import by @yuneng-berri in #35802
chore: remove unused .flake8 config and flake8 dev dependency by @mateo-berri in #35888
chore: stop advising pre-commit and bootstrap by @mateo-berri in #35884
fix(auth): name enable_jwt_auth when a JWT-shaped key is rejected by @yassin-berriai in #35831
feat(auto-router): make reminder marker pair configurable by @akapur99 in #35874
fix(UI): update anthropic model presets by @tin-berri in #35896
fix(bootstrap): switch to the dashboard node floor via nvm or fnm by @mateo-berri in #35895
perf(pre-commit): run python, dashboard, and gen-api checks concurrently by @mateo-berri in #35903
feat(spend): derive a default auto-router savings baseline from the hardest tier by @tin-berri in #35907
fix(http_handler): self-heal handler clients closed after cache eviction by @mateo-berri in #35862
fix(cost_tracking): keep OpenAI prompt cache token details through usage reassembly by @devin-ai-integration[bot] in #34812
fix(cost): bill gpt-5.6 prompt cache reads at the cache read rate by @devin-ai-integration[bot] in #34957
fix(batches): account for Responses API usage by @rimysore in #35367
ci: retry Codecov uploads and stop failing jobs on OIDC token flakes by @mateo-berri in #35251
feat(complexity_router): let operators rename the four complexity tiers by @akapur99 in #35893
chore(lint): zero stale ruff and LIT headroom and strip inert type: ignore comments by @mateo-berri in #35928
chore(lint): zero out seven more purely local basedpyright rules by @mateo-berri in #35927
chore(ui): zero stale headroom on local dashboard eslint budgets by @mateo-berri in #35929
fix(managed-files): skip rows without file objects by @rimysore in #35365
fix(router): redact fallback tracebacks at the call site and cover the sync deferred stream by @yassin-berriai in #35843
fix(migrations): recover from an interrupted Prisma toolchain install by @yassin-berriai in #35832
fix(lint): bring basedpyright rule counts back under their budget limits by @mateo-berri in #35962
chore(ui): don't zero out stale headroom except no-console by @mateo-berri in #35964
fix(proxy): give proxy_admin_viewer read parity with proxy_admin by @ryan-crabbe-berri in #35851
refactor(ui): address UI lint budget issues by refactoring UI by @tin-berri in #35960
fix(ci): make the env-key doc gate see get_secret_bool reads by @yassin-berriai in #35833
fix(caching): re-land evicted LLM client closing (#35492) atop self-healing handlers by @mateo-berri in #35870
fix(proxy): keep the connected DB client when a startup health check fails by @yassin-berriai in #35837
chore(lint): remove litellm/types from the ruff lint exclusion by @mateo-berri in #35926
feat(sgr): make the gateway middleware the source of truth for successful requests by @yassin-berriai in #35717
feat(auto-router): let operators replace the LLM classifier's system prompt by @akapur99 in #35855
fix(docker): bake the pip image's prisma engines at a world-readable path by @yassin-berriai in #35976
fix(auth): return 403 from the OAuth2 enterprise gate by @yassin-berriai in #35838
fix(router): keep custom model_info across a price data reload by @yassin-berriai in #35491
fix(proxy): resolve pass-through credentials live from router deployments by @mateo-berri in #35916
fix(ci): fetch only head and merge-base in lint jobs instead of every branch by @mateo-berri in #35982
fix(autorouter): match CJK keyword_tier_rules that regex word boundaries miss by @akapur99 in #35984
feat(spend): rebuild the auto-router benchmarks backend as a per-session rollup by @tin-berri in #35910
refactor(ui): replace hand-rolled query-param routing with nuqs by @ryan-crabbe-berri in #35871
fix(docker): bake the componentized prisma engines at /opt/prisma so any uid can start by @yassin-berriai in #35989
fix(migrations): keep the toolchain heal from raising on an unreadable nodeenv cache by @yassin-berriai in #35986
fix(bedrock): sign Bedrock managed-file S3 requests with S3SigV4Auth by @yassin-berriai in #35983
chore(typing): replace Any seams with real types across responses, proxy, and provider adapters by @mateo-berri in #35809
fix(ai21): resolve the documented AI21_API_KEY instead of a misspelled name by @yassin-berriai in #35985
fix(docker): fail the image build when the generated prisma engine paths drift off /opt/prisma by @yassin-berriai in #35979
fix(jina_ai): resolve the documented JINA_API_KEY as a fallback by @yassin-berriai in #35992
fix(proxy): only treat a recoverable database outage as grounds to serve without one by @yassin-berriai in #35864
fix(ci): make every remaining CI checkout shallow by @mateo-berri in #35997
fix(auto-router): stop the embedding model's context window from failing long requests by @akapur99 in #35956
fix(ci): make the env-key doc gate see bare get_secret and get_secret_str reads by @yassin-berriai in #35996
fix(logging): extend secret redaction to records litellm does not emit directly by @yassin-berriai in #35977
test(utils): pin the register_model replay test to the recorded half by @yassin-berriai in #35994
fix(ci): run every helm test suite, not just the first one per file by @yassin-berriai in #35993
ci: fail the build when a test file or Dockerfile is invoked by no job by @yassin-berriai in #35991
fix(langfuse): stop a collected httpx handler from closing a shared client by @yassin-berriai in #35981
fix(bedrock): grant bedrock:CountTokens in OIDC session policy by @devin-ai-integration[bot] in #33145
feat(pre-commit): save full lint output to a per-worktree log file by @mateo-berri in #36004
feat(ui): match auto-router preset models against deployments' underlying model IDs by @tin-berri in #35972
fix(core_helpers): map generic 'error' finish_reason to 'stop' by @devin-ai-integration[bot] in #33972
fix(proxy)!: apply request-parameter checks consistently across body, path and form inputs by @yuneng-berri in #36011
fix: rebuild models_by_provider in add_known_models so cost map reloads reach wildcard expansion by @ryan-crabbe-berri in #36010
feat(complexity_router): report LLM classifier cost per request via routing_decision and x-litellm-classifier-cost header by @tin-berri in #36015
fix(model-prices): correct replicate model key typo by @AkashNaickar in #34800
fix(proxy): register managed batch output files on terminal retrieve by @Souravrajvi0 in #34092
perf(pre-commit): fetch basedpyright base counts from CI artifacts by @mateo-berri in #35970
fix(ui): sync projects list page index to ?page= so back and reload keep the page by @ryan-crabbe-berri in #36003
fix(ui): link project page keys to their virtual key detail by @ryan-crabbe-berri in #36002
refactor(ui): drop unreferenced locals from dashboard route components by @yuneng-berri in #35819
fix(ui): opening a project now pushes ?project= so back and deep links work by @ryan-crabbe-berri in #36001
refactor(ui): drop unreferenced locals from shared dashboard components by @yuneng-berri in #35821
refactor(ui): drop unreferenced locals from tests and narrow destructures by @yuneng-berri in #36025
fix(guardrails): allow litellm_content_filter to run on post_mcp_call by @mateo-berri in #35980
fix(guardrails): scan /v1/messages tool traffic by @mateo-berri in #35999
refactor(ui): drop dead locals and unused React state across the dashboard by @yuneng-berri in #36026
feat(ui): add the auto-router usage tab to cost optimization by @tin-berri in #35995
fix(managed_files): derive unified output file ids deterministically so concurrent registrations converge by @mateo-berri in #36019
fix(proxy): send keepalive pings on anthropic messages SSE streams during upstream silence by @mateo-berri in #36024
fix(managed_files): return unified ids from unscoped file listing by @mateo-berri in #36031
fix(arize_phoenix): lowercase OTLP/gRPC auth metadata key by @devin-ai-integration[bot] in #34883
fix(auto-router): accept every reminder marker pair a harness emits by @tin-berri in #36029
fix(pricing): sync flex/priority tier keys to dated OpenAI snapshot variants by @mateo-berri in #35923
fix(cost): bill reasoning tokens at the service tier output rate by @mateo-berri in #35925
fix(proxy): include today's UTC bucket when a daily activity range ends at the caller's current day by @tin-berri in #36051
fix: expired-miss share over all measured turns + cost-optimization tab labels by @tin-berri in #36037
fix(router): include Bedrock batch/S3 fields and model in deployment credentials by @mpcusack-altos in #24548
fix(batch): track cost for managed batches with no attributable key/u… by @elinacse in #35468
feat(guardrails): add scan_only_tool_results to scope unified guardrails to tool results by @mateo-berri in #36014
fix(cost): stop token-pricing the placeholder input on file content calls by @devin-ai-integration[bot] in #35140
fix(proxy): fetch background responses through the router in CheckResponsesCost by @devin-ai-integration[bot] in #35137
fix(proxy): yaml store_prompts_in_spend_logs should take precedence over DB cached value by @Praveena-617 in #35769
fix(lint): measure the basedpyright budget gate in a gate-owned venv by @mateo-berri in #36050
docs: cap all GitHub comments at 15-25 words, curb semicolon splices by @mateo-berri in #36059
chore(lint): name MappingProxyType in the mutable-collection fix messages by @mateo-berri in #36072
test: roll back runtime model registrations between tests by @mateo-berri in #36039
refactor(types): cut 653 implicit and explicit Any diagnostics across 11 modules by @mateo-berri in #36054
fix(proxy): stop resolving the UI session sentinel team on /search_tools/list by @yuneng-berri in #36061
fix(batches): persist managed file ids for cancelled/failed/expired batches by @mateo-berri in #36048
fix(batches): register managed output files on batch cancel by @mateo-berri in #36034
fix(proxy): allow non-admins to reach /user/daily/activity/aggregated by @yuneng-berri in #36062
fix(anthropic): coerce explicit additionalProperties to false in output_format schema by @dkindlund in #35811
fix(batches): prevent managed file fallbacks by @rimysore in #35371
chore: ignore the mechanical lint and typing sweeps in git blame by @mateo-berri in #36076
fix(proxy): warn at startup when max_budget is set but no database is connected by @devin-ai-integration[bot] in #36041
fix(proxy): promote caller metadata trace fields into litellm_metadata by @yucheng-berri in #35866
feat(terraform): sync provider 0.3.0 from the mirror and cut 0.4.0 by @yuneng-berri in #36098
fix(guardrails): honor configured timeout in Zscaler AI Guard by @yucheng-berri in #36110
fix(logging): fall back to litellm_metadata when metadata is empty by @yucheng-berri in #36105
fix(proxy): re-assert the authenticated identity on passthrough requests by @yucheng-berri in #36121
chore: bump litellm-enterprise 0.1.53 -> 0.1.54, litellm-proxy-extras 0.4.83 -> 0.4.84 by @yuneng-berri in #36139
fix(ui): match auto-router preset models against wildcard-expanded model groups by @tin-berri in #36111
test(router): assert the auto-router max_input_chars kwarg by @yuneng-berri in #36109
fix(ui): allow clearing a key's budget reset from the Edit Key form by @ryan-crabbe-berri in #36140
fix(managed_files): skip unparseable rows when listing managed files by @mateo-berri in #36021
fix(a2a): stop writing per-caller headers onto the shared cached httpx client by @yassin-berriai in #35978
build(deps): bump h2 to 4.4.1 and js-yaml to 4.3.1 by @yuneng-berri in #36147
chore: promote staging to main by @mateo-berri in #36057

New Contributors

@rimysore made their first contribution in #35367
@AkashNaickar made their first contribution in #34800
@Souravrajvi0 made their first contribution in #34092
@elinacse made their first contribution in #35468

Full Changelog: v1.97.0-dev.1...v1.97.0-dev.2