Installation
pip install openllm==0.2.17To upgrade from a previous version, use the following command:
pip install --upgrade openllm==0.2.17Usage
All available models: openllm models
To start a LLM: python -m openllm start opt
To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it --entrypoint=/bin/bash -P ghcr.io/bentoml/openllm:0.2.17 openllm --help
Find more information about this release in the CHANGELOG.md
What's Changed
- feat: optimize model saving and loading on single GPU by @aarnphm in #183
- fix(ci): update version correctly [skip ci] by @aarnphm in #184
- fix(models): setup xformers in base container and loading PyTorch meta weights by @aarnphm in #185
- infra(generation): initial work for generating tokens by @aarnphm in #186
- ci: pre-commit autoupdate [pre-commit.ci] by @pre-commit-ci in #187
- feat: --force-push to allow force push to bentocloud by @aarnphm in #188
Full Changelog: v0.2.16...v0.2.17