BerriAI/litellm v1.57.10

on GitHub

v1.57.10 - Alpha Release

🚨 This is an alpha release - we've made several performance / RPS improvements to litellm core. If you see any issues please file it https://github.com/BerriAI/litellm/issues

• Litellm dev 01 10 2025 p2 by @krrishdholakia in #7679
• Litellm dev 01 10 2025 p3 by @krrishdholakia in #7682
• build: new ui build by @krrishdholakia in #7685
• fix(model_hub.tsx): clarify cost in model hub is per 1m tokens by @krrishdholakia in #7687
• Litellm dev 01 11 2025 p3 by @krrishdholakia in #7702
• (perf litellm) - use _get_model_info_helper for cost tracking by @ishaan-jaff in #7703
• (perf sdk) - minor changes to cost calculator to run helpers only when necessary by @ishaan-jaff in #7704
• (perf) - proxy, use orjson for reading request body by @ishaan-jaff in #7706
• (minor fix - aiohttp_openai/) - fix get_custom_llm_provider by @ishaan-jaff in #7705
• (sdk perf fix) - only print args passed to litellm when debugging mode is on by @ishaan-jaff in #7708
• (perf) - only use response_cost_calculator 1 time per request. (Don't re-use the same helper twice per call ) by @ishaan-jaff in #7709
• [BETA] Add OpenAI /images/variations + Topaz API support by @krrishdholakia in #7700
• (litellm sdk speedup router) - adds a helper _cached_get_model_group_info to use when trying to get deployment tpm/rpm limits by @ishaan-jaff in #7719

Full Changelog: v1.57.8...v1.57.10

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.57.10

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results