BerriAI/litellm v1.63.14-nightly

on GitHub

What's Changed

• Modify completion handler for SageMaker to use payload from prepared_request by @andjsmi in #9326
• Arize integration Fix by @nate-mar in #9338
• Fix get_llm_provider error + add model name to tpm/rpm cache key (enables wildcard models to work w/ usage-based-routing) by @krrishdholakia in #9355
• Litellm dev 03 18 2025 p1 v3 by @krrishdholakia in #9354
• [Bug Fix] Arize AI Logging Integration with LiteLLM by @ishaan-jaff in #9352
• build(model_prices_and_context_window.json): fix azure gpt-4o pricing by @krrishdholakia in #9361
• Contributor PR: Fix sagemaker too little data for content error by @krrishdholakia in #9335
• [Feat] - API - Allow using dynamic Arize AI Spaces on LiteLLM by @ishaan-jaff in #9353
• fix(common_utils.py): handle cris only model by @krrishdholakia in #9363
• docs(litellm_proxy): correct parameter assignment in litellm proxy docs by @colesmcintosh in #9375
• Feature flag checking LiteLLM_CredentialsTable by @krrishdholakia in #9376
• fix(lowest_tpm_rpm_v2.py): support batch writing increments to redis by @krrishdholakia in #9357
• Support 'prisma migrate' for db schema changes by @krrishdholakia in #9382
• Fix latency redis by @emerzon in #9387
• Revert "Fix latency redis" by @krrishdholakia in #9388
• build(model_prices_and_context_window.json): add o1-pro pricing by @krrishdholakia in #9397
• [Bug Fix] - Azure OpenAI - ensure SSL verification runs by @ishaan-jaff in #9341
• [Feat] - Allow building custom prompt management integration by @ishaan-jaff in #9384
• Litellm fix icons by @azdolinski in #9374
• [UI Improvement] Use local icons for model providers instead of downloading them by @ishaan-jaff in #9399
• fix(internal_user_endpoints.py): re-introduce upsert on user not found by @krrishdholakia in #9395
• docs: Include Phoenix Page into sidebar under observability by @SrilakshmiC in #9332
• fix(types/utils.py): support openai 'file' message type by @krrishdholakia in #9402
• feat: Add support for custom OPENROUTER_API_BASE via get_secret in co… by @graysonchen in #9369
• fix: VertexAI outputDimensionality configuration by @JamesGuthrie in #9366
• docs(vertexai): fix typo in required env variables by @Shiti in #9325
• Update perplexity.md by @jollyolliel in #9290
• fix: VoyageAI prompt_token always empty by @lucasra1 in #9260
• build(deps): bump litellm from 1.55.3 to 1.61.15 in /cookbook/litellm-ollama-docker-image by @dependabot in #9422
• [Feat] OpenAI o1-pro Responses API streaming support by @ishaan-jaff in #9419
• [Feat] Add OpenAI o1-pro support on Responses API by @ishaan-jaff in #9415
• [Docs - Draft] LiteLLM x MCP Interface by @ishaan-jaff in #9435
• support returning api-base on pass-through endpoints + consistently return 404 if team not found in DB by @krrishdholakia in #9439
• fix(handle_error.py): make cooldown error more descriptive by @krrishdholakia in #9438
• Consistent anthropic response_format streaming/non-streaming behaviour by @krrishdholakia in #9437
• New Azure Models (GPT-4.5-Preview, Mistral Small 3.1) by @emerzon in #9453
• Set max size limit to in-memory cache item - prevents OOM errors by @krrishdholakia in #9448
• fix(model_param_helper.py): update _get_litellm_supported_transcription_kwargs() to use proper annotations from TranscriptionCreateParamsNonStreaming & ``TranscriptionCreateParamsStreaming` by @hsaeed3 in #9451
• [Feat] LiteLLM x MCP Bridge - Use MCP Tools with LiteLLM by @ishaan-jaff in #9436
• fix(model_param_helper.py): update _get_litellm_supported_transcription_kwargs() to use proper annotations from TranscriptionCreateParamsNonStreaming & `TranscriptionCreateParamsStreaming by @ishaan-jaff in #9455

New Contributors

• @andjsmi made their first contribution in #9326
• @azdolinski made their first contribution in #9374
• @SrilakshmiC made their first contribution in #9332
• @graysonchen made their first contribution in #9369
• @JamesGuthrie made their first contribution in #9366
• @Shiti made their first contribution in #9325
• @jollyolliel made their first contribution in #9290
• @hsaeed3 made their first contribution in #9451

Full Changelog: v1.63.12-nightly...v1.63.14-nightly

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.14-nightly

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results