What's Changed
- Fix transcription model name mapping by @colesmcintosh in #11333
- [Feat] DD Trace - Add instrumentation for streaming chunks by @ishaan-jaff in #11338
- UI - Custom Server Root Path (Multiple Fixes) by @krrishdholakia in #11337
- [Perf] - Add Async + Batched S3 Logging by @ishaan-jaff in #11340
- fixes: expose flag to disable token counter by @ishaan-jaff in #11344
- Merge in - Gemini streaming - thinking content parsing - return in
reasoning_contentby @krrishdholakia in #11298
Full Changelog: v1.72.0.dev1...v1.72.0.dev3
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.72.0.dev3
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 220.0 | 236.14372402204256 | 6.217513201371485 | 0.0 | 1860 | 0 | 197.26605100004235 | 1261.3509589999694 |
| Aggregated | Passed ✅ | 220.0 | 236.14372402204256 | 6.217513201371485 | 0.0 | 1860 | 0 | 197.26605100004235 | 1261.3509589999694 |