Installation
pip install openllm==0.3.0To upgrade from a previous version, use the following command:
pip install --upgrade openllm==0.3.0Usage
All available models: openllm models
To start a LLM: python -m openllm start opt
To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P ghcr.io/bentoml/openllm:0.3.0 start opt
To run OpenLLM Clojure UI (community-maintained): docker run -p 8420:80 ghcr.io/bentoml/openllm-ui-clojure:0.3.0
Find more information about this release in the CHANGELOG.md
What's Changed
- cron(style): run formatter [generated] [skip ci] by @aarnphm in #257
- feat(vllm): streaming by @aarnphm in #260
- chore(build): use latest vllm pre-built kernel by @aarnphm in #261
- chore(deps): bump bentoml/setup-bentoml-action from 59beefe94e2e8f8ebbedf555fc86bd5d1ae0a708 to d4010d8303684b183f45b33c6a44644f81337bdb by @dependabot in #267
- chore(deps): bump actions/checkout from 3.5.3 to 3.6.0 by @dependabot in #269
- ci: pre-commit autoupdate [pre-commit.ci] by @pre-commit-ci in #273
- chore(deps): bump docker/setup-buildx-action from 2.9.1 to 2.10.0 by @dependabot in #271
- chore(deps): bump aws-actions/configure-aws-credentials from 2.2.0 to 3.0.1 by @dependabot in #270
- chore(deps): bump taiki-e/install-action from 2.16.0 to 2.17.2 by @dependabot in #268
- chore(deps): bump @mui/icons-material from 5.14.3 to 5.14.6 by @dependabot in #265
- chore(deps-dev): bump shadow-cljs from 2.25.2 to 2.25.3 by @dependabot in #262
- chore(deps): bump @mui/base from 5.0.0-beta.11 to 5.0.0-beta.12 by @dependabot in #264
- chore(deps): bump @mui/material from 5.14.5 to 5.14.6 by @dependabot in #266
- chore(deps): bump @mui/x-date-pickers from 6.11.2 to 6.12.0 by @dependabot in #263
- infra: update build dependencies [clojure-ui build] by @aarnphm in #278
- fix: persistent styling between ruff and yapf by @aarnphm in #279
- Fixed weird characters over model dropdowns by @GutZuFusss in #280
- refactor(breaking): unify LLM API by @aarnphm in #283
- fix(yapf): align weird new lines break [generated] [skip ci] by @aarnphm in #284
- fix(serving): vllm distributed size by @aarnphm in #285
- chore(deps): bump sigstore/cosign-installer from 3.1.1 to 3.1.2 by @dependabot in #296
- chore(deps): bump taiki-e/install-action from 2.17.2 to 2.17.8 by @dependabot in #295
- chore(deps): bump aquasecurity/trivy-action from 559eb1224e654a86c844a795e6702a0742c60c72 to fbd16365eb88e12433951383f5e99bd901fc618f by @dependabot in #294
- chore(deps): bump github/codeql-action from 2.21.4 to 2.21.5 by @dependabot in #292
- chore(deps): bump crazy-max/ghaction-import-gpg from 5.3.0 to 5.4.0 by @dependabot in #293
- fix(gptq): use upstream integration by @aarnphm in #297
- ci: pre-commit autoupdate [pre-commit.ci] by @pre-commit-ci in #298
- chore(deps): bump @mui/x-date-pickers from 6.12.0 to 6.12.1 by @dependabot in #291
- chore(deps): bump @mui/icons-material from 5.14.6 to 5.14.7 by @dependabot in #290
- chore(deps): bump @mui/material from 5.14.6 to 5.14.7 by @dependabot in #289
- chore(deps): bump @mui/base from 5.0.0-beta.12 to 5.0.0-beta.13 by @dependabot in #287
- chore(deps): bump @mui/x-data-grid from 6.11.2 to 6.12.1 by @dependabot in #288
Full Changelog: v0.2.27...v0.3.0