BerriAI/litellm v1.70.1-stable

on GitHub

What's Changed

• Handle gemini audio input by @krrishdholakia in #10739
• Fixed Ollama Structured Response not working #10616 by @imdigitalashish in #10617
• fix(factory.py): Add reasoning content handling for missing assistant… by @LouisShark in #10688
• [Feat] Add tools support for Nvidia NIM by @ishaan-jaff in #10763
• [Fix]: /messages - allow using dynamic AWS params by @ishaan-jaff in #10769
• fix: pass application/json for GenericAPILogger by @ishaan-jaff in #10772
• [Docs] Using litellm with Google ADK by @ishaan-jaff in #10777
• Update Nscale model providers to point to website by @OscarSavNS in #10764
• [Fix] Allow using dynamic aws_region with /messages on Bedrock by @ishaan-jaff in #10779
• [Feat] Option to force/always use the litellm proxy (#10559) (#10633) by @ishaan-jaff in #10773
• feat: Addded EU Anthropic Inference profile for Claude 3.7 by @wagnerjt in #10767
• Add new model provider Novita AI (#7582) by @krrishdholakia in #9527
• Support Anthropic web search tool + Add more google finish reason mapping by @krrishdholakia in #10785
• Fix azure dall e 3 call with custom model name + Handle Bearer $LITELLM_API_KEY in x-litellm-api-key custom header by @krrishdholakia in #10776
• [Refactor] Move LLM Guard, Secret Detection to Enterprise Pip packagea by @ishaan-jaff in #10782
• fix: URL encode OTEL_EXPORTER_OTLP_TRACES_HEADERS for Phoenix Integration by @arizedatngo in #10654
• [README] Add Llama API in supported providers table by @seyeong-han in #10802
• [Fix] Maximum Vertex Recursion Depth Reached when transforming response_format by @ishaan-jaff in #10798
• add ollama api base defaults by @jugaldb in #10805
• Add Azure gpt-4o-mini-tts by @emerzon in #10807
• Add Azure Cohere Embed v4 by @emerzon in #10806
• [Fix] Allow overriding all constants using a .env variable by @ishaan-jaff in #10803
• [Bug Fix] Presidio integration failing and making inference return 500 by @ishaan-jaff in #10197
• [Feat] Allow specifying PII Entities Config when using Presidio Guardrails by @ishaan-jaff in #10810
• [Bedrock] Fixes to only allow accepted fields for tool json schema by @ishaan-jaff in #10062
• Fix: Separate Sync/Async Azure Client Cache to Prevent Type Mismatches by @daikeren in #10781
• fix(litellm_logging.py): log custom headers in requester metadata by @krrishdholakia in #10818
• Migrate cohere to /v2/embed - adds support for output_dimensions param by @krrishdholakia in #10809
• add support to parse metadata by @jugaldb in #10832
• Team id + Status filter on logs by @NANDINI-star in #10831
• Update Groq models on model_prices_and_context_window.json by @naliotopier in #10077
• [Feat] Add endpoints for adding, deleting, editing guardrails in DB by @ishaan-jaff in #10833
• [Feat UI] Add Guardrails on LiteLLM UI by @ishaan-jaff in #10834
• [UI] Fixes for bulk invite users - add validation for bad inputs by @ishaan-jaff in #10836
• Support OTLP standard environment variables for configuration by @damienpontifex in #10813
• [Feat] Presidio Improvements - Allow adding presidio api base on UI, Test presidio on Test Key, fixes for running presidio hook by @ishaan-jaff in #10840
• patch: validate_team_org_change by @ishaan-jaff in #10835
• Feat add retention config by @jugaldb in #10815
• fix failing test typo by @jugaldb in #10843
• Capture Gemini reasoning tokens usage in streaming mode by @Dima-Mediator in #10789
• Anthropic unified web search + tool cost tracking support by @krrishdholakia in #10846
• avoid error AttributeError: 'Message' object has no attribute 'audio' by @igtm in #10814
• fix: bedrock-sonnet-caching-cost by @shibaboy in #10752
• Fix/issue 10113 embeddings use non default tokenizer by @camfarineau in #10629
• Add models import command by @msabramo in #10581
• Add documentation on in-memory and disk caching by @user202729 in #10522
• Bedrock pixtral fix by @ajac-zero in #10439
• added fix to resolve delete API by @jugaldb in #10845
• build(deps): bump undici from 6.21.1 to 6.21.3 in /docs/my-website by @dependabot in #10855
• [Refactor] Make Pagerduty a free feature by @ishaan-jaff in #10857
• [Refactor] Move enterprise_routes within litellm_enterprise by @ishaan-jaff in #10860
• refactor: make send_management_endpoint_alert a free feature by @ishaan-jaff in #10863
• [Feat - Guardrails] Expose /apply_guardrail endpoint for directly calling guardrail by @ishaan-jaff in #10867
• [CI/CD] Test: fix mapped tests by @ishaan-jaff in #10870
• [Feat] UI: Add support for Deleting Guardrail on UI by @ishaan-jaff in #10869
• [UI] Allow adding Bedrock, Presidio, Lakera, AIM guardrails on UI by @ishaan-jaff in #10874
• fix(proxy/_types.py): add missing comma for /v2/rerank by @krrishdholakia in #10861
• fix(streaming_handler.py): patch vertex ai meta llama streaming error by @krrishdholakia in #10878
• build(dependencies)📦: Add numpydoc to dependencies in pyproject.toml by @ericmjl in #10736
• [Feat]: Configure LiteLLM to Parse User Headers from Open Web UI by @damgem in #9802
• Feat/support anonymize in aim guardrail by @hxdror in #10757
• Gemini Multimodal Live API support by @krrishdholakia in #10841
• Feat log retention cleanup by @jugaldb in #10872
• Model filter added on logs tab. (#10877) by @ishaan-jaff in #10891
• fix: fix novita ai docs by @jasonhp in #10882
• [Feat] Add Tracing for guardrails in StandardLoggingPayload, Langfuse by @ishaan-jaff in #10890
• [UI] - Add Guardrail Tracing to LiteLLM SpendLogs by @ishaan-jaff in #10893
• Fix example commands in reliability.md by @wwwillchen in #10894
• Feat: add guardrail tracing to OTEL, Arize phoenix by @ishaan-jaff in #10896
• Key hash filter added by @NANDINI-star in #10903
• [Feat] UI: Allow clicking into a guardrail by @ishaan-jaff in #10904
• [Feat] Add Lakera v2 Guardrail Support by @ishaan-jaff in #10880
• Key alias filter on logs by @NANDINI-star in #10905
• [Feat] handle multi-pod deployment for SpendLogs Retention by @jugaldb in #10895
• [UI] Fixes for using Lakera Guardrail on LiteLLM UI by @ishaan-jaff in #10906
• add constant to config settings.md by @jugaldb in #10908
• Remove extra test file by @jugaldb in #10889
• [UI] Allow editing guardrails by @ishaan-jaff in #10907
• Handle openai gpt file data + add openai 'supports_pdf_input' to all vision models + Support bedrock tool cache pointing by @krrishdholakia in #10897

New Contributors

• @imdigitalashish made their first contribution in #10617
• @LouisShark made their first contribution in #10688
• @OscarSavNS made their first contribution in #10764
• @arizedatngo made their first contribution in #10654
• @jugaldb made their first contribution in #10805
• @daikeren made their first contribution in #10781
• @naliotopier made their first contribution in #10077
• @damienpontifex made their first contribution in #10813
• @Dima-Mediator made their first contribution in #10789
• @igtm made their first contribution in #10814
• @shibaboy made their first contribution in #10752
• @camfarineau made their first contribution in #10629
• @ajac-zero made their first contribution in #10439
• @damgem made their first contribution in #9802
• @hxdror made their first contribution in #10757
• @wwwillchen made their first contribution in #10894

Full Changelog: v1.69.0-stable...v1.70.1-stable