🍱 BentoML has just released v1.0.6 featuring the gRPC preview! Without changing a line of code, you can now serve your Bentos as a gRPC service. Similar to serving over HTTP, BentoML gRPC supports all the ML frameworks, observability features, adaptive batching, and more out-of-the-box, simply by calling the serve-grpc CLI command.
> pip install "bentoml[grpc]"
> bentoml serve-grpc iris_classifier:latest --production- Checkout our updated tutorial for a quick 10-minute crash course of BentoML gRPC.
- Review the standardized Protobuf definition of service APIs and IO types, NDArray, DataFrame, File/Image, JSON, etc.
- Learn more about multi-language client support (Python, Go, Java, Node.js, etc) with working examples.
- Customize gRPC service by mounting new servicers and interceptors.
⚠️ gRPC is current under preview. The public APIs may undergo incompatible changes in the future patch releases until the official v1.1.0 minor version release.
- Enhanced access logging format to output Trace and Span IDs in the more standard hex encoding by default.
- Added request total, duration, and in-progress metrics to Runners, in addition to API Servers.
- Added support for XGBoost SKLearn models.
- Added support for restricting image mime types in the Image IO descriptor.
🥂 We’d like to thank our community for their contribution and support.
- Shout out to @benjamintanweihao for fixing a BentoML CLI bug.
- Shout out to @lsh918 for mixing a PyTorch framework issue.
- Shout out to @jeffthebear for enhancing the Pandas DataFrame OpenAPI schema.
- Shout out to @jiewpeng for adding the support for customizing access logs with Trace and Span ID formats.
What's Changed
- fix: log runner errors explicitly by @ssheng in #2952
- ci: temp fix for models test by @sauyon in #2949
- fix: fix context parameter for multi-input IO descriptors by @sauyon in #2948
- fix: use
torch.from_numpy()instead oftorch.Tensor()to keep data type by @lsh918 in #2951 - docs: fix wrong name for example neural net by @ssun-g in #2959
- docs: fix bentoml containerize command help message by @aarnphm in #2957
- chore(cli): remove unused
--no-truncby @benjamintanweihao in #2965 - fix: relax regex for setting environment variables by @benjamintanweihao in #2964
- docs: update wrong paths for disabling logs by @creativedutchmen in #2974
- feat: track serve update for start subcommands by @ssheng in #2976
- feat: logging customization by @jiewpeng in #2961
- chore(cli): using quotes instead of backslash by @sauyon in #2981
- feat(cli): show full tracebacks in debug mode by @sauyon in #2982
- feature(runner): add multiple output support by @larme in #2912
- docs: add airflow integration page by @parano in #2990
- chore(ci): fix the unit test of transformers by @bojiang in #3003
- chore(ci): fix the issue caused by the change of check_task by @bojiang in #3004
- fix(multipart): support multipart file inputs to non-file descriptors by @sauyon in #3005
- feat(server): add runner metrics; refactoring batch size metrics by @bojiang in #2977
- EXPERIMENTAL: gRPC support by @aarnphm in #2808
- fix(runner): receive requests before cork by @bojiang in #2996
- fix(server): service_name label of runner metrics by @bojiang in #3008
- chore(misc): remove mentioned for team member from PR request by @aarnphm in #3009
- feat(xgboost): support xgboost sklearn models by @sauyon in #2997
- feat(io/image): allow restricting mime types by @sauyon in #2999
- fix(grpc): docker message by @aarnphm in #3012
- fix: broken legacy metrics by @aarnphm in #3019
- fix(e2e): exception test for image IO by @aarnphm in #3017
- revert(3017): filter write-only mime type for Image IO by @bojiang in #3020
- chore: cleanup containerize utils by @aarnphm in #3014
- feat(proto): add
serialized_bytestopb.Partby @aarnphm in #3022 - docs: Update README.md by @parano in #3023
- chore(grpc): vcs generated stubs by @aarnphm in #3016
- feat(io/image): allow writeable mimes as output by @sauyon in #3024
- docs: fix descriptor typo by @darioarias in #3027
- fix(server): log localhost instead of 0.0.0.0 by @sauyon in #3033
- fix(io): Pandas OpenAPI schema by @jeffthebear in #3032
- chore(docker): support more cuda versions by @larme in #3035
- docs: updates on blocks that failed to render by @aarnphm in #3031
- chore: migrate to pyproject.toml by @aarnphm in #3025
- docs: gRPC tutorial by @aarnphm in #3013
- docs: gRPC advanced guides by @aarnphm in #3034
- feat(configuration): override options with envvar by @bojiang in #3018
- chore: update links by @aarnphm in #3040
- fix(configuration): should validate config early by @aarnphm in #3041
- qa(bentos): update latest options by @aarnphm in #3042
- qa: ignore tools from distribution by @aarnphm in #3045
- dependencies: ignore broken pypi combination by @aarnphm in #3043
- feat: gRPC tracking by @aarnphm in #3015
- configuration: migrate schema to
api_serverby @ssheng in #3046 - qa: cleanup MLflow by @aarnphm in #2945
New Contributors
- @lsh918 made their first contribution in #2951
- @ssun-g made their first contribution in #2959
- @benjamintanweihao made their first contribution in #2965
- @creativedutchmen made their first contribution in #2974
- @darioarias made their first contribution in #3027
- @jeffthebear made their first contribution in #3032
Full Changelog: v1.0.5...v1.0.6