bentoml/OpenLLM v0.1.19

on GitHub

Models

MPT supported, with its fine-tune and pre-trained variants

Fixes bugs for loading local modules, address some loading bugs

OpenLLM now tentitatively releases binary distribution for MacOS, Windows, and Linux

openllm.LLMConfig now supports dict() protocol

config = openllm.LLMConfig.for_model("opt")

print(config.items())
print(config.values())
print(config.keys())
print(dict(config))

See #85

Installation

pip install openllm==0.1.19

To upgrade from a previous version, use the following command:

pip install --upgrade openllm==0.1.19

Usage

All available models: python -m openllm.models

To start a LLM: python -m openllm start dolly-v2

Find more information about this release in the CHANGELOG.md

Full Changelog: v0.1.18...v0.1.19