bentoml/BentoML v1.0.20

BentoML - v1.0.20

on GitHub

🍱 BentoML v1.0.20 is released with improved usability and compatibility features.

• Production Mode by Default: bentoml serve command will now run with the --production option by default. The change is made the simulate the production behavior during development. The --reload option will continue to with as expected. To achieve the serving behavior previously, use --development instead.

• Optional Dependency for OpenTelemetry Exporter: The opentelemetry-exporter-otlp-proto-http dependency has been moved from a required dependency to an optional one to address a protobuf dependency incompatibility issue. ⚠️ If you are currently using the Model Monitoring and Inference Data Collection feature, you must install the package with the monitor-otlp ****option from this release onwards to include the necessary dependency.

  pip install "bentoml[monitor-otlp]"

• OpenTelemetry Trace ID Configuration Option: A new configuration option has been added to return the OpenTelemetry Trace ID in the response. This feature is particularly helpful when tracing has not been initialized in the upstream caller, but the caller still wishes to log the Trace ID in case of an error.

  api_server:
    http:
      response:
        trace_id: True

• Start from a Service: Added the ability to start a server from a bentoml.Service object. This is helpful for troubleshooting a project in a development environment where no Bentos has been built yet.

  import bentoml
  
  # import the Service defined in `/clip_api_service/service.py` file
  from clip_api_service.service import svc 
  
  if __name__ == "__main__":
    # start a server:
    server = bentoml.HTTPServer(svc)
    server.start(blocking=False)
    client = server.get_client()
    client.predict(..)

What's Changed

• fix(dispatcher): handling empty o_stat in trigger_refresh by @larme in #3796
• fix(framework): adjust diffusers device_map default behavior by @larme in #3779
• chore(dispatcher): cancel jobs with a for loop by @sauyon in #3788
• fix: correctly reraise CancelledError by @sauyon in #3801
• use path as resource for non-OS paths by @sauyon in #3800
• chore(deps): bump coverage[toml] from 7.2.3 to 7.2.4 by @dependabot in #3803
• feat: embedded runner by @larme in #3735
• feat(tensorflow): support list types inputs by @enmanuelmag in #3807
• chore(deps): bump ruff from 0.0.263 to 0.0.264 by @dependabot in #3817
• feat: subprocess build by @aarnphm in #3814
• docs: update community slack links by @parano in #3824
• chore(deps): bump pyarrow from 11.0.0 to 12.0.0 by @dependabot in #3820
• chore(deps): remove imageio by @aarnphm in #3812
• chore(deps): bump tritonclient[all] from 2.32.0 to 2.33.0 by @dependabot in #3795
• ci: add Pillow to tests dependencies by @aarnphm in #3830
• feat(observability): support service.name by @aarnphm in #3825
• feat: optional returning trace_id in response by @aarnphm in #3827
• chore: 3.11 support by @PeterJCLaw in #3792
• fix: Eliminate the exception during shutdown by @frostming in #3826
• chore: expose scheduling_strategy in to_runner by @bojiang in #3831
• feat: allow starting server with bentoml.Service instance by @parano in #3829
• chore(deps): bump bufbuild/buf-setup-action from 1.17.0 to 1.18.0 by @dependabot in #3838
• fix: make sure to set content-type for file type by @aarnphm in #3837
• docs: update default docs to use env as key:value instead of list type by @aarnphm in #3841
• deps: move exporter-proto to optional by @aarnphm in #3840
• feat(server): improve server APIs by @aarnphm in #3834

New Contributors

• @enmanuelmag made their first contribution in #3807
• @PeterJCLaw made their first contribution in #3792

Full Changelog: v1.0.19...v1.0.20