What's Changed
- (UI) Delete Internal Users on Admin UI by @ishaan-jaff in #6442
- (testing) increase prometheus.py test coverage to 90% by @ishaan-jaff in #6466
- (Feat) New Logging integration - add Datadog LLM Observability support by @ishaan-jaff in #6449
- (feat) add pricing for amazon.titan-embed-image-v1 by @ishaan-jaff in #6444
- LiteLLM Minor Fixes & Improvements (10/24/2024) by @krrishdholakia in #6441
- Litellm dev 10 26 2024 by @krrishdholakia in #6472
- (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected by @ishaan-jaff in #6471
- redis otel tracing + async support for latency routing by @krrishdholakia in #6452
- (fix) Prometheus - Log Postgres DB latency, status on prometheus by @ishaan-jaff in #6484
- (router_strategy/) ensure all async functions use async cache methods by @ishaan-jaff in #6489
- (fix) proxy - fix when
STORE_MODEL_IN_DB
should be set by @ishaan-jaff in #6492 - (fix)
PrometheusServicesLogger
_get_metric
should return metric in Registry by @ishaan-jaff in #6486 - Add
azure/gpt-4o-mini-2024-07-18
to model_prices_and_context_window… by @xingyaoww in #6477 - Update utils.py by @vibhanshu-ob in #6468
New Contributors
- @xingyaoww made their first contribution in #6477
- @vibhanshu-ob made their first contribution in #6468
Full Changelog: v1.51.0...v1.51.1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
---|---|---|---|---|---|---|---|---|---|
/chat/completions | Passed ✅ | 200.0 | 217.96900764879396 | 6.232037758758388 | 0.0 | 1865 | 0 | 178.63059899997324 | 1220.8741560000362 |
Aggregated | Passed ✅ | 200.0 | 217.96900764879396 | 6.232037758758388 | 0.0 | 1865 | 0 | 178.63059899997324 | 1220.8741560000362 |