BerriAI/litellm v1.70.1-stable on GitHub

What's Changed

Handle gemini audio input by @krrishdholakia in #10739
Fixed Ollama Structured Response not working #10616 by @imdigitalashish in #10617
fix(factory.py): Add reasoning content handling for missing assistant… by @LouisShark in #10688
[Feat] Add tools support for Nvidia NIM by @ishaan-jaff in #10763
[Fix]: /messages - allow using dynamic AWS params by @ishaan-jaff in #10769
fix: pass application/json for GenericAPILogger by @ishaan-jaff in #10772
[Docs] Using litellm with Google ADK by @ishaan-jaff in #10777
Update Nscale model providers to point to website by @OscarSavNS in #10764
[Fix] Allow using dynamic aws_region with /messages on Bedrock by @ishaan-jaff in #10779
[Feat] Option to force/always use the litellm proxy (#10559) (#10633) by @ishaan-jaff in #10773
feat: Addded EU Anthropic Inference profile for Claude 3.7 by @wagnerjt in #10767
Add new model provider Novita AI (#7582) by @krrishdholakia in #9527
Support Anthropic web search tool + Add more google finish reason mapping by @krrishdholakia in #10785
Fix azure dall e 3 call with custom model name + Handle Bearer $LITELLM_API_KEY in x-litellm-api-key custom header by @krrishdholakia in #10776
[Refactor] Move LLM Guard, Secret Detection to Enterprise Pip packagea by @ishaan-jaff in #10782
fix: URL encode OTEL_EXPORTER_OTLP_TRACES_HEADERS for Phoenix Integration by @arizedatngo in #10654
[README] Add Llama API in supported providers table by @seyeong-han in #10802
[Fix] Maximum Vertex Recursion Depth Reached when transforming response_format by @ishaan-jaff in #10798
add ollama api base defaults by @jugaldb in #10805
Add Azure gpt-4o-mini-tts by @emerzon in #10807
Add Azure Cohere Embed v4 by @emerzon in #10806
[Fix] Allow overriding all constants using a .env variable by @ishaan-jaff in #10803
[Bug Fix] Presidio integration failing and making inference return 500 by @ishaan-jaff in #10197
[Feat] Allow specifying PII Entities Config when using Presidio Guardrails by @ishaan-jaff in #10810
[Bedrock] Fixes to only allow accepted fields for tool json schema by @ishaan-jaff in #10062
Fix: Separate Sync/Async Azure Client Cache to Prevent Type Mismatches by @daikeren in #10781
fix(litellm_logging.py): log custom headers in requester metadata by @krrishdholakia in #10818
Migrate cohere to /v2/embed - adds support for output_dimensions param by @krrishdholakia in #10809
add support to parse metadata by @jugaldb in #10832
Team id + Status filter on logs by @NANDINI-star in #10831
Update Groq models on model_prices_and_context_window.json by @naliotopier in #10077
[Feat] Add endpoints for adding, deleting, editing guardrails in DB by @ishaan-jaff in #10833
[Feat UI] Add Guardrails on LiteLLM UI by @ishaan-jaff in #10834
[UI] Fixes for bulk invite users - add validation for bad inputs by @ishaan-jaff in #10836
Support OTLP standard environment variables for configuration by @damienpontifex in #10813
[Feat] Presidio Improvements - Allow adding presidio api base on UI, Test presidio on Test Key, fixes for running presidio hook by @ishaan-jaff in #10840
patch: validate_team_org_change by @ishaan-jaff in #10835
Feat add retention config by @jugaldb in #10815
fix failing test typo by @jugaldb in #10843
Capture Gemini reasoning tokens usage in streaming mode by @Dima-Mediator in #10789
Anthropic unified web search + tool cost tracking support by @krrishdholakia in #10846
avoid error AttributeError: 'Message' object has no attribute 'audio' by @igtm in #10814
fix: bedrock-sonnet-caching-cost by @shibaboy in #10752
Fix/issue 10113 embeddings use non default tokenizer by @camfarineau in #10629
Add models import command by @msabramo in #10581
Add documentation on in-memory and disk caching by @user202729 in #10522
Bedrock pixtral fix by @ajac-zero in #10439
added fix to resolve delete API by @jugaldb in #10845
build(deps): bump undici from 6.21.1 to 6.21.3 in /docs/my-website by @dependabot in #10855
[Refactor] Make Pagerduty a free feature by @ishaan-jaff in #10857
[Refactor] Move enterprise_routes within litellm_enterprise by @ishaan-jaff in #10860
refactor: make send_management_endpoint_alert a free feature by @ishaan-jaff in #10863
[Feat - Guardrails] Expose /apply_guardrail endpoint for directly calling guardrail by @ishaan-jaff in #10867
[CI/CD] Test: fix mapped tests by @ishaan-jaff in #10870
[Feat] UI: Add support for Deleting Guardrail on UI by @ishaan-jaff in #10869
[UI] Allow adding Bedrock, Presidio, Lakera, AIM guardrails on UI by @ishaan-jaff in #10874
fix(proxy/_types.py): add missing comma for /v2/rerank by @krrishdholakia in #10861
fix(streaming_handler.py): patch vertex ai meta llama streaming error by @krrishdholakia in #10878
build(dependencies)📦: Add numpydoc to dependencies in pyproject.toml by @ericmjl in #10736
[Feat]: Configure LiteLLM to Parse User Headers from Open Web UI by @damgem in #9802
Feat/support anonymize in aim guardrail by @hxdror in #10757
Gemini Multimodal Live API support by @krrishdholakia in #10841
Feat log retention cleanup by @jugaldb in #10872
Model filter added on logs tab. (#10877) by @ishaan-jaff in #10891
fix: fix novita ai docs by @jasonhp in #10882
[Feat] Add Tracing for guardrails in StandardLoggingPayload, Langfuse by @ishaan-jaff in #10890
[UI] - Add Guardrail Tracing to LiteLLM SpendLogs by @ishaan-jaff in #10893
Fix example commands in reliability.md by @wwwillchen in #10894
Feat: add guardrail tracing to OTEL, Arize phoenix by @ishaan-jaff in #10896
Key hash filter added by @NANDINI-star in #10903
[Feat] UI: Allow clicking into a guardrail by @ishaan-jaff in #10904
[Feat] Add Lakera v2 Guardrail Support by @ishaan-jaff in #10880
Key alias filter on logs by @NANDINI-star in #10905
[Feat] handle multi-pod deployment for SpendLogs Retention by @jugaldb in #10895
[UI] Fixes for using Lakera Guardrail on LiteLLM UI by @ishaan-jaff in #10906
add constant to config settings.md by @jugaldb in #10908
Remove extra test file by @jugaldb in #10889
[UI] Allow editing guardrails by @ishaan-jaff in #10907
Handle openai gpt file data + add openai 'supports_pdf_input' to all vision models + Support bedrock tool cache pointing by @krrishdholakia in #10897

New Contributors

@imdigitalashish made their first contribution in #10617
@LouisShark made their first contribution in #10688
@OscarSavNS made their first contribution in #10764
@arizedatngo made their first contribution in #10654
@jugaldb made their first contribution in #10805
@daikeren made their first contribution in #10781
@naliotopier made their first contribution in #10077
@damienpontifex made their first contribution in #10813
@Dima-Mediator made their first contribution in #10789
@igtm made their first contribution in #10814
@shibaboy made their first contribution in #10752
@camfarineau made their first contribution in #10629
@ajac-zero made their first contribution in #10439
@damgem made their first contribution in #9802
@hxdror made their first contribution in #10757
@wwwillchen made their first contribution in #10894

Full Changelog: v1.69.0-stable...v1.70.1-stable