What's Changed
- gemini price updates: gemma 3, flash 2 thinking update, learnlm by @yigitkonur in #9190
- Mark Cohere Embedding 3 models as Multimodal by @emerzon in #9176
- Fix Metadata not updating in Team UI by @lucasra1 in #9180
- feat: initial commit adding support for credentials on proxy ui by @krrishdholakia in #9186
- Fix azure ai services url + add azure data zone pricing by @krrishdholakia in #9185
- (gemini)Handle HTTP 201 status code in Vertex AI response by @youngchannelforyou in #9193
- feat/postgres-volumes by @xucailiang in #8741
- [FEAT] Support for Snowflake REST API LLMs #7979 by @SunnyWan59 in #8950
- fix(azure.py): track azure llm api latency metric by @krrishdholakia in #9217
- Support bedrock converse cache token tracking by @krrishdholakia in #9221
- Emit audit logs on All user + model Create/Update/Delete endpoints by @krrishdholakia in #9223
- (UI Usage) - Allow clicking into Top Keys when showing users Top API Key by @ishaan-jaff in #9225
- [Feat] Add Snowflake Cortex to LiteLLM by @ishaan-jaff in #9222
- [Fixes] Responses API - allow /responses and subpaths as LLM API route + Add exception mapping for responses API by @ishaan-jaff in #9220
- docs: Add centralized credential management docs by @bexelbie in #9254
- Docs: Update configs.md by @bexelbie in #9263
- Support reusing existing model credentials by @krrishdholakia in #9267
- LiteLLM UI Fixes by @krrishdholakia in #9269
- Fix "system" role has become unacceptable in ollama by @briandevvn in #9261
- Litellm rc 03 14 2025 patch 1 by @krrishdholakia in #9271
- [Feat] UI - Add Test Connection by @ishaan-jaff in #9272
- [UI] Fix 1 - instantly show newly create keys on Admin UI (don't require refresh) by @ishaan-jaff in #9257
- (UI) Fix model edit + delete - instantly show edit + deletes to models by @ishaan-jaff in #9258
New Contributors
- @lucasra1 made their first contribution in #9180
- @youngchannelforyou made their first contribution in #9193
- @xucailiang made their first contribution in #8741
- @SunnyWan59 made their first contribution in #8950
- @bexelbie made their first contribution in #9254
- @briandevvn made their first contribution in #9261
Full Changelog: v1.63.8-nightly...v1.63.11-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.11-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 240.0 | 272.07713013991287 | 6.161766827537479 | 0.0 | 1844 | 0 | 213.46316099999285 | 4315.458979000027 |
| Aggregated | Passed ✅ | 240.0 | 272.07713013991287 | 6.161766827537479 | 0.0 | 1844 | 0 | 213.46316099999285 | 4315.458979000027 |