Installation
pip install openllm==0.3.7To upgrade from a previous version, use the following command:
pip install --upgrade openllm==0.3.7Usage
All available models: openllm models
To start a LLM: python -m openllm start opt
To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P ghcr.io/bentoml/openllm:0.3.7 start opt
To run OpenLLM Clojure UI (community-maintained): docker run -p 8420:80 ghcr.io/bentoml/openllm-ui-clojure:0.3.7
Find more information about this release in the CHANGELOG.md
What's Changed
- chore(deps): bump taiki-e/install-action from 2.18.9 to 2.18.13 by @dependabot in #372
- chore(deps): bump pypa/cibuildwheel from 2.15.0 to 2.16.0 by @dependabot in #376
- chore(deps): bump github/codeql-action from 2.21.5 to 2.21.8 by @dependabot in #378
- chore(deps): bump @mui/material from 5.14.8 to 5.14.9 by @dependabot in #368
- chore(deps-dev): bump @edge-runtime/vm from 3.1.0 to 3.1.3 by @dependabot in #370
- chore(deps): bump @floating-ui/dom from 1.5.1 to 1.5.3 by @dependabot in #366
- chore(deps-dev): bump @typescript-eslint/parser from 6.4.1 to 6.7.0 by @dependabot in #365
- chore(deps): bump @vercel/og from 0.5.11 to 0.5.13 by @dependabot in #356
- chore(deps-dev): bump postcss from 8.4.23 to 8.4.29 by @dependabot in #360
- chore(deps-dev): bump turbo from 1.10.13 to 1.10.14 by @dependabot in #364
- chore(bench): add more prompts by @aarnphm in #380
- chore(deps): bump @mui/styled-engine from 5.13.2 to 5.14.10 by @dependabot in #379
- chore(deps): bump @mui/base from 5.0.0-beta.14 to 5.0.0-beta.16 by @dependabot in #384
- chore(deps): bump @babel/runtime from 7.22.10 to 7.22.15 by @dependabot in #362
- chore(deps): bump @mui/system from 5.14.8 to 5.14.10 by @dependabot in #383
- chore(deps): bump @mui/x-data-grid from 6.13.0 to 6.14.0 by @dependabot in #361
- chore(deps): bump @mui/private-theming from 5.14.8 to 5.14.10 by @dependabot in #382
- chore(deps): bump @mui/icons-material from 5.14.8 to 5.14.9 by @dependabot in #367
- chore(deps): bump @mui/utils from 5.14.8 to 5.14.10 by @dependabot in #381
- ci: pre-commit autoupdate [pre-commit.ci] by @pre-commit-ci in #406
- chore(deps): bump actions/checkout from 4.0.0 to 4.1.0 by @dependabot in #405
- chore(deps): bump taiki-e/install-action from 2.18.13 to 2.18.16 by @dependabot in #404
- chore(deps-dev): bump eslint from 8.47.0 to 8.50.0 by @dependabot in #400
- chore(deps): bump nextra from 2.11.1 to 2.13.0 by @dependabot in #393
- chore(deps-dev): bump @types/react from 18.2.20 to 18.2.22 by @dependabot in #403
- chore(deps-dev): bump tailwindcss from 3.3.2 to 3.3.3 by @dependabot in #398
- chore(deps-dev): bump autoprefixer from 10.4.12 to 10.4.16 by @dependabot in #397
- chore(deps): bump @vercel/og from 0.5.13 to 0.5.17 by @dependabot in #396
- chore(deps-dev): bump @next/eslint-plugin-next from 13.4.19 to 13.5.2 by @dependabot in #392
- chore(deps): bump @floating-ui/core from 1.4.1 to 1.5.0 by @dependabot in #402
- chore(deps-dev): bump @svgr/webpack from 2.0.0-alpha.26fa501a to 8.1.0 by @dependabot in #389
- chore(deps): bump nextra-theme-docs from 2.12.3 to 2.13.1 by @dependabot in #411
- chore(deps-dev): bump @typescript-eslint/eslint-plugin from 6.7.0 to 6.7.3 by @dependabot in #414
- chore(deps-dev): bump shadow-cljs from 2.25.4 to 2.25.6 by @dependabot in #394
- chore(deps): bump @mui/x-data-grid from 6.14.0 to 6.15.0 by @dependabot in #395
- chore(deps-dev): bump @typescript-eslint/parser from 6.7.0 to 6.7.3 by @dependabot in #412
- chore(deps-dev): bump @types/react from 18.2.20 to 18.2.23 by @dependabot in #413
- chore(deps-dev): bump postcss from 8.4.29 to 8.4.30 by @dependabot in #390
- chore(deps): bump @mui/x-data-grid from 6.15.0 to 6.16.0 by @dependabot in #434
- chore(deps-dev): bump postcss from 8.4.30 to 8.4.31 by @dependabot in #430
- chore(deps): bump pypa/cibuildwheel from 2.16.0 to 2.16.1 by @dependabot in #440
- chore(deps): bump github/codeql-action from 2.21.8 to 2.21.9 by @dependabot in #439
- chore(deps): bump taiki-e/install-action from 2.18.16 to 2.19.2 by @dependabot in #438
- chore(deps): bump @mui/x-date-pickers from 6.13.0 to 6.16.0 by @dependabot in #428
- chore(deps-dev): bump @types/react from 18.2.20 to 18.2.24 by @dependabot in #427
- chore(deps-dev): bump @types/node from 20.5.3 to 20.8.0 by @dependabot in #435
- ci: pre-commit autoupdate [pre-commit.ci] by @pre-commit-ci in #441
- chore(deps): bump @babel/runtime from 7.22.15 to 7.23.1 by @dependabot in #436
- chore(deps): bump nextra from 2.13.0 to 2.13.1 by @dependabot in #425
- feat: PromptTemplate and system prompt support by @MingLiangDai in #407
- feat: OpenAI compatible API by @XunchaoZ in #417
- chore(deps): bump pypa/cibuildwheel from 2.16.1 to 2.16.2 by @dependabot in #474
- chore(deps): bump github/codeql-action from 2.21.9 to 2.22.0 by @dependabot in #473
- chore(deps): bump aws-actions/configure-aws-credentials from 4.0.0 to 4.0.1 by @dependabot in #471
- chore(deps): bump actions/setup-python from 4.7.0 to 4.7.1 by @dependabot in #470
- chore(deps): bump taiki-e/install-action from 2.19.2 to 2.20.1 by @dependabot in #472
- ci: pre-commit autoupdate [pre-commit.ci] by @pre-commit-ci in #475
- fix: do not reply on env var for built bento/docker by @larme in #477
- fix: asyncio stalling inside notebooks by @xianml in #478
- fix: fix client HTTPS by @sauyon in #480
- feat: add 1 openllm llama2 notebook demo by @xianml in #479
- chore(deps-dev): bump eslint from 8.47.0 to 8.51.0 by @dependabot in #469
- chore(deps-dev): bump @types/node from 20.5.3 to 20.8.3 by @dependabot in #467
- chore(deps-dev): bump turbo from 1.10.14 to 1.10.15 by @dependabot in #461
- chore(deps): bump nextra from 2.13.1 to 2.13.2 by @dependabot in #458
- chore(deps): bump nextra-theme-docs from 2.13.1 to 2.13.2 by @dependabot in #457
- chore(deps-dev): bump @typescript-eslint/eslint-plugin from 6.7.3 to 6.7.5 by @dependabot in #483
- chore(deps): bump highlight.js from 11.8.0 to 11.9.0 by @dependabot in #456
- chore(deps-dev): bump @next/eslint-plugin-next from 13.5.2 to 13.5.4 by @dependabot in #464
- chore(deps-dev): bump @typescript-eslint/parser from 6.7.3 to 6.7.5 by @dependabot in #482
- chore(deps-dev): bump prettier-plugin-tailwindcss from 0.5.4 to 0.5.5 by @dependabot in #455
- chore(deps-dev): bump @types/react-transition-group from 4.4.6 to 4.4.7 by @dependabot in #460
- chore(deps-dev): bump @edge-runtime/vm from 3.1.3 to 3.1.4 by @dependabot in #466
- chore(deps): bump github/codeql-action from 2.22.0 to 2.22.2 by @dependabot in #484
- chore(deps): bump taiki-e/install-action from 2.20.1 to 2.20.2 by @dependabot in #485
- chore(deps-dev): bump @types/react from 18.2.20 to 18.2.28 by @dependabot in #486
- chore(deps): bump @mui/x-data-grid from 6.16.0 to 6.16.2 by @dependabot in #487
- chore(deps): bump @vercel/og from 0.5.17 to 0.5.18 by @dependabot in #488
- chore(deps-dev): bump @types/node from 20.5.3 to 20.8.4 by @dependabot in #491
- chore(deps): bump @babel/runtime from 7.23.1 to 7.23.2 by @dependabot in #490
- feat(client): simple implementation and streaming by @aarnphm in #256
New Contributors
- @MingLiangDai made their first contribution in #407
- @XunchaoZ made their first contribution in #417
- @xianml made their first contribution in #478
- @sauyon made their first contribution in #480
Full Changelog: v0.3.6...v0.3.7