Installation
pip install openllm==0.4.2To upgrade from a previous version, use the following command:
pip install --upgrade openllm==0.4.2Usage
All available models: openllm models
To start a LLM: python -m openllm start opt
To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P ghcr.io/bentoml/openllm:0.4.2 start opt
To run OpenLLM Clojure UI (community-maintained): docker run -p 8420:80 ghcr.io/bentoml/openllm-ui-clojure:0.4.2
Find more information about this release in the CHANGELOG.md
What's Changed
- refactor(cli): cleanup API by @aarnphm in #592
- infra: move out clojure to external by @aarnphm in #593
- infra: using ruff formatter by @aarnphm in #594
- infra: remove tsconfig by @aarnphm in #595
- revert: configuration not to dump flatten by @aarnphm in #597
- package: add openllm core dependencies to labels by @aarnphm in #600
- fix: loading correct local models by @aarnphm in #599
- fix: correct importmodules locally by @aarnphm in #601
- fix: overload flattened dict by @aarnphm in #602
- feat(client): support authentication token and shim implementation by @aarnphm in #605
- fix(client): check for should retry header by @aarnphm in #606
- chore(client): remove ununsed state enum by @aarnphm in #609
- chore: remove generated stubs for now by @aarnphm in #610
- refactor(config): simplify configuration and update start CLI output by @aarnphm in #611
- docs: update supported feature set by @aarnphm in #612
Full Changelog: v0.4.1...v0.4.2