Full Changelog: v1.73.6.rc-draft...v1.73.6-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.73.6-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 190.0 | 210.86720644249243 | 6.277304583930227 | 0.0 | 1878 | 0 | 166.350910999995 | 6235.332546000023 |
| Aggregated | Passed ✅ | 190.0 | 210.86720644249243 | 6.277304583930227 | 0.0 | 1878 | 0 | 166.350910999995 | 6235.332546000023 |
What's Changed
- fix(proxy): Fix test_mock_create_audio_file by adding managed_files hook by @colesmcintosh in #12072
- Enhance CircleCI integration in LLM translation testing workflow by @colesmcintosh in #12041
- Inkeep searchbar and chat added to the Docs by @NANDINI-star in #12030
- [Fix] Redis - Add better debugging to see what variables are set by @ishaan-jaff in #12073
- Fix today selector date mutation bug in dashboard components by @colesmcintosh in #12042
- Responses API - Add reasoning content support for non-OpenAI providers by @ryan-castner in #12055
- Raise clearer error on anthropic unified route + add new
new_keyparam for regenerating key by @krrishdholakia in #12087 - Refactor: bedrock passthrough fixes - migrate to Passthrough SDK by @krrishdholakia in #12089
- Fix Azure-OpenAI Vision API Compliance by @davis-featherstone in #12075
- [Bug Fix] Bedrock Guardrails - Ensure PII Masking is applied on response streaming or non streaming content when using post call by @ishaan-jaff in #12086
- fix(docs): Remove unused dotenv dependency from docusaurus config by @colesmcintosh in #12102
- [Fix] MCP - Ensure internal users can access /mcp and /mcp/ routes by @ishaan-jaff in #12106
- fix: handle provider_config type error in passthrough error handler by @colesmcintosh in #12101
- Add o3 and o4-mini deep research models by @krrishdholakia in #12109
- [Bug Fix] Anthropic - Token Usage Null Handling in calculate_usage by @Gum-Joe in #12068
- fix: change cost calculation logs from INFO to DEBUG level by @colesmcintosh in #12112
- fix: set logger levels based on LITELLM_LOG environment variable by @colesmcintosh in #12111
- [Feat] Add Bridge from generateContent <> /chat/completions by @ishaan-jaff in #12081
- [Docs] - Show how to use fallbacks with audio transcriptions endpoints by @ishaan-jaff in #12115
- [Bug Fix] Fix handling str, bool types for
mock_testing_fallbackson router using /audio endpoints by @ishaan-jaff in #12117 - Adding Feature: Palo Alto Networks Prisma AIRS Guardrail by @jroberts2600 in #12116
- [Bug Fix] Exception mapping for context window exceeded - should catch anthropic exceptions by @ishaan-jaff in #12113
- docs(GEMINI.md): add development guidelines and architecture overview by @colesmcintosh in #12035
- [Bug fix] Router - handle cooldown_time = 0 for deployments by @ishaan-jaff in #12108
- [Feat] Add Eleven Labs - Speech To Text Support on LiteLLM by @ishaan-jaff in #12119
- Revert "fix: set logger levels based on LITELLM_LOG environment variable" by @ishaan-jaff in #12122
- Fix Braintrust integration: Adds model to metadata to calculate cost and corrects docs by @ohmeow in #12022
- [Fix] Change Message init type annotation to support other roles by @amarrella in #11942
- Add "Get Code" Feature by @NANDINI-star in #11629
- Bedrock Passthrough cost tracking (
/invoke+/converseroutes - streaming + non-streaming) by @krrishdholakia in #12123 - feat: add local LLM translation testing with artifact generation by @colesmcintosh in #12120
- [Feat] introduce new environment variable NO_REDOC to opt-out Redoc by @zhangyoufu in #12092
- Fix user-team association issues in LiteLLM proxy by @colesmcintosh in #12082
- feat: enhance redaction functionality for EmbeddingResponse by @bougou in #12088
- De-duplicate models in team settings dropdown by @NANDINI-star in #12074
- Add Azure OpenAI assistant features cost tracking by @colesmcintosh in #12045
- Remove duplicated entry in logs on key cache update by @Mte90 in #12032
- Update model_prices_and_context_window.json by @codeugar in #11972
- Litellm batch api background cost calc by @krrishdholakia in #12125
- Selecting 'test connect' resets the public model name when selecting an azure model by @NANDINI-star in #11713
- [Bug Fix] Invite links email should contain the correct invite id by @ishaan-jaff in #12130
- fix example config.yaml in claude code tutorial by @glgh in #12133
New Contributors
- @ryan-castner made their first contribution in #12055
- @davis-featherstone made their first contribution in #12075
- @Gum-Joe made their first contribution in #12068
- @jroberts2600 made their first contribution in #12116
- @ohmeow made their first contribution in #12022
- @amarrella made their first contribution in #11942
- @zhangyoufu made their first contribution in #12092
- @bougou made their first contribution in #12088
- @codeugar made their first contribution in #11972
- @glgh made their first contribution in #12133
Full Changelog: v1.73.2-nightly...v1.73.6-nightly