What's Changed
- (docs improvement) remove emojis, use
guides
section, categorize uncategorized docs by @ishaan-jaff in #6796 - (docs) simplify left nav names + use a section for
making llm requests
by @ishaan-jaff in #6799 - Bump cross-spawn from 7.0.3 to 7.0.5 in /ui by @dependabot in #6779
- Docs - use 1 page for all logging integrations on proxy + add logging features at top level by @ishaan-jaff in #6805
- (docs) add docstrings for all /key, /user, /team, /customer endpoints by @ishaan-jaff in #6804
- LiteLLM Minor Fixes & Improvements (11/15/2024) by @krrishdholakia in #6746
- (Proxy) add support for DOCS_URL and REDOC_URL by @ishaan-jaff in #6806
- feat - add
fireworks_ai/qwen2p5-coder-32b-instruct
by @ishaan-jaff in #6818 - Litellm stable pr 10 30 2024 by @krrishdholakia in #6821
- (Feat) Add provider specific budget routing by @ishaan-jaff in #6817
- (feat) provider budget routing improvements by @ishaan-jaff in #6827
Full Changelog: v1.52.10...v1.52.11
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.11
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Failed ❌ | 270.0 | 300.82403009385007 | 6.198177352347725 | 0.0 | 1854 | 0 | 229.45128300000306 | 3106.586268000001 |
Aggregated | Failed ❌ | 270.0 | 300.82403009385007 | 6.198177352347725 | 0.0 | 1854 | 0 | 229.45128300000306 | 3106.586268000001 |