bentoml/BentoML v1.1.11

BentoML - v1.1.11

on GitHub

Bug fixes

• Fix streaming for long payloads on remote runners. It will now always yield text and follow SSE protocol. We also provide SSE utils:

import bentoml
from bentoml.io import SSE

class MyRunnable(bentoml.Runnable):
	@bentoml.Runnable.method()
	def streaming(self, text):
		yield "data: 1\n\n"
		yield "data: 12222222222222222222222222222\n\n"

runner = bentoml.Runner(MyRunnable)

svc = bentoml.Service("service", runners=[runner])

@svc.api()
def infer(text):
	result = 0
	async for it in runner.streaming.async_stream(text):
		payload = SSE.from_iterator(it)
		result += int(payload.data)
	return result

What's Changed

• docs: Add BentoCloud payment doc by @Sherlock113 in #4286
• docs: update quickstart with OpenLLM by @aarnphm in #4295
• fix(docs): correct server implementation by @aarnphm in #4297
• docs: Remove bill void status by @Sherlock113 in #4299
• docs: Update LLM quickstart format and wording by @Sherlock113 in #4300
• fix: citation link at README.md by @shenxiangzhuang in #4301
• fix(transformers): support trust_remote_code and added unit tests by @MingLiangDai in #4271
• docs: Add Bento Deployment details docs by @Sherlock113 in #4304
• test: remove outdated tests with pretrained_class parameter by @MingLiangDai in #4308
• fix: Updated starlette to >= 0.24.0 by @jakthra in #4306
• ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in #4316
• fix: syntax error on code snippet in bentoml.onnx.save_model docs by @lucasew in #4323
• chore(deps): bump actions/setup-python from 4 to 5 by @dependabot in #4329
• docs: fixed typo in file name benchmark README.md by @gazon1 in #4342
• fix(stream): streaming enable to work with proxy by @jianshen92 in #4330
• docs: fix typo in frameworks transformers guide by @IbrahimAmin1 in #4360
• chore(sse): refactor sse utils with efficient buffering by @bojiang in #4362
• fix(runner): fix DataFrame container header too long by @larme in #4364
• chore(generated): new stubs for proto 4 by @aarnphm in #4374

New Contributors

• @shenxiangzhuang made their first contribution in #4301
• @jakthra made their first contribution in #4306
• @lucasew made their first contribution in #4323
• @gazon1 made their first contribution in #4342
• @IbrahimAmin1 made their first contribution in #4360

Full Changelog: v1.1.10...v1.1.11