bentoml/BentoML v1.0.0-rc3 on GitHub

We have just released BentoML 1.0.0rc3 with a number of highly anticipated features and improvements. Check it out with the following command!

$ pip install -U bentoml --pre

⚠️ BentoML will release the official 1.0.0 version next week and remove the need to use --pre tag to install BentoML versions after 1.0.0. If you wish to stay on the 0.13.1 LTS version, please lock the dependency with bentoml==0.13.1.

Added support for framework runners in the following ML frameworks.
Added support for Huggingface Transformers custom pipelines.
Fixed a logging issue causing the api_server and runners to not generate error logs.
Optimized Tensorflow inference procedure.

Improved resource request configuration for runners.

Resource request can be now configured in the BentoML configuration. If unspecified, runners will be scheduled to best utilized the available system resources.
```
runners:
  resources:
    cpu: 8.0
    nvidia.com/gpu: 4.0
```

Updated the API for custom runners to declare the types of supported resources.

import bentoml

class MyRunnable(bentoml.Runnable):
	SUPPORTS_CPU_MULTI_THREADING = True  # Deprecated SUPPORT_CPU_MULTI_THREADING
        SUPPORTED_RESOURCES = ("nvidia.com/gpu", "cpu")  # Deprecated SUPPORT_NVIDIA_GPU
        ...

my_runner = bentoml.Runner(
    MyRunnable,
    runnable_init_params={"foo": foo, "bar": bar},
    name="custom_runner_name",
    ...
)

Deprecated the API for specifying resources from the framework to_runner() and custom Runner APIs. For better flexibility at runtime, it is recommended to specifying resources through configuration.

What's Changed

fix(dependencies): require pyyaml>=5 by @sauyon in #2626
refactor(server): merge contexts; add yatai headers by @bojiang in #2621
chore(pylint): update pylint configuration by @sauyon in #2627
fix: Transformers NVIDIA_VISIBLE_DEVICES value type casting by @ssheng in #2624
fix: Server silently crash without logging exceptions by @ssheng in #2635
fix(framework): some GPU related fixes by @larme in #2637
tests: minor e2e test cleanup by @sauyon in #2643
docs: Add model in bentoml.pytorch.save_model() pytorch integration example by @AlexandreNap in #2644
chore(ci): always enable actions on PR by @sauyon in #2646
chore: updates ci by @aarnphm in #2650
fix(docker): templates bash heredoc should pass -ex by @aarnphm in #2651
feat: CatBoost integration by @yetone in #2615
feat: FastAI by @aarnphm in #2571
feat: Support Transformers custom pipeline by @ssheng in #2640
feat(framework): onnx support by @larme in #2629
chore(tensorflow): optimize inference procedure by @bojiang in #2567
fix(runner): validate runner names by @sauyon in #2588
fix(runner): lowercase runner names and add tests by @sauyon in #2656
style: github naming by @aarnphm in #2659
tests(framework): add new framework tests by @sauyon in #2660
docs: missing code annotation by @jjmachan in #2654
perf(templates): cache python installation via conda by @aarnphm in #2662
fix(ci): destroy the runner after init_local by @bojiang in #2665
fix(conda): python installation order by @aarnphm in #2668
fix(tensorflow): casting error on kwargs by @bojiang in #2664
feat(runner): implement resource configuration by @sauyon in #2632

New Contributors

@AlexandreNap made their first contribution in #2644

Full Changelog: v1.0.0-rc2...v1.0.0-rc3

bentoml/BentoML v1.0.0-rc3 BentoML - 1.0.0-rc3 on GitHub

What's Changed

New Contributors

bentoml/BentoML v1.0.0-rc3
BentoML - 1.0.0-rc3

on GitHub