Installation
pip install openllm==0.3.14To upgrade from a previous version, use the following command:
pip install --upgrade openllm==0.3.14Usage
All available models: openllm models
To start a LLM: python -m openllm start opt
To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P ghcr.io/bentoml/openllm:0.3.14 start opt
To run OpenLLM Clojure UI (community-maintained): docker run -p 8420:80 ghcr.io/bentoml/openllm-ui-clojure:0.3.14
Find more information about this release in the CHANGELOG.md
What's Changed
- chore(deps): bump taiki-e/install-action from 2.20.15 to 2.21.3 by @dependabot in #546
- ci: pre-commit autoupdate [pre-commit.ci] by @pre-commit-ci in #548
- chore(deps): bump aquasecurity/trivy-action from 0.12.0 to 0.13.0 by @dependabot in #545
- chore(deps): bump github/codeql-action from 2.22.4 to 2.22.5 by @dependabot in #544
- fix: update llama2 notebook example by @xianml in #516
- chore(deps-dev): bump @types/react from 18.2.20 to 18.2.33 by @dependabot in #542
- chore(deps-dev): bump @typescript-eslint/eslint-plugin from 6.8.0 to 6.9.0 by @dependabot in #537
- chore(deps-dev): bump @edge-runtime/vm from 3.1.4 to 3.1.6 by @dependabot in #540
- chore(deps-dev): bump eslint from 8.47.0 to 8.52.0 by @dependabot in #541
- fix: Max new tokens by @XunchaoZ in #550
- chore(inference): update vllm to 0.2.1.post1 and update config parsing by @aarnphm in #554
Full Changelog: v0.3.13...v0.3.14