Full Changelog: v1.70.1.dev2...v1.70.2-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.2-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 480.0 | 551.6473700786378 | 5.693462012007638 | 0.0 | 1704 | 0 | 435.9962309999901 | 1522.4978100000044 |
| Aggregated | Failed ❌ | 480.0 | 551.6473700786378 | 5.693462012007638 | 0.0 | 1704 | 0 | 435.9962309999901 | 1522.4978100000044 |
What's Changed
- add docs for spend logs by @jugaldb in #10913
- [QA/Guardrails] Allow editing guardrail entity configs by @ishaan-jaff in #10918
- [UI QA Guardrails] - Minor UI Fixes by @ishaan-jaff in #10920
- [CI/CD] Fix - linting cryptography error by @ishaan-jaff in #10921
- LiteLLM Stable release notes by @krrishdholakia in #10919
- docs apply guardrail endpoint by @ishaan-jaff in #10923
- [Docs] Pii masking with LiteLLM by @ishaan-jaff in #10924
- Add provider Featherless AI (access to 4200+ models) by @DarinVerheijke in #10596
- fix(vertex_llm_base.py): remove quota_project_id parameter from crede… by @krrishdholakia in #10915
- Enable structured JSON schema outputs in LM Studio by @estsauver in #10929
- update sambanova models and parameters by @jhpiedrahitao in #10900
- Fix Cohere Rerank Provider by @keyute in #10822
- add skip server startup flag to cli by @mohittalele in #10665
- Allow passed in vertex_ai credentials to be authorized_user type by @pselden in #10899
- fix: handle DB_USER, DB_PASSWORD, DB_HOST problem I faced, since this… by @unrealandychan in #10842
- Add keys and members count in team info by @NANDINI-star in #10950
- Update github.md by @dastaiger in #10946
- docs: add llms.txt and llms_full.txt for LLM-friendly scraping by @colesmcintosh in #10961
- [Fix] Invitation Email does not include the invitation link by @ishaan-jaff in #10958
- Enable key reassignment on UI + Show 'updated at' column for key in all keys table by @krrishdholakia in #10960
- Update existing model's model info on UI + Show model access groups when creating keys/teams on UI by @krrishdholakia in #10965
- Validate migrating keys to teams + Fix mistral image url on async translation by @krrishdholakia in #10966
- add cla to docs by @jugaldb in #10963
- [Fix] List Guardrails - Show config.yaml guardrails on litellm ui by @ishaan-jaff in #10959
- fix: vertex show clear exception on failed refresh by @ishaan-jaff in #10969
New Contributors
- @DarinVerheijke made their first contribution in #10596
- @estsauver made their first contribution in #10929
- @mohittalele made their first contribution in #10665
- @pselden made their first contribution in #10899
- @unrealandychan made their first contribution in #10842
- @dastaiger made their first contribution in #10946
Full Changelog: v1.70.1-stable...v1.70.2-nightly