Release Note (3.18.0)
Release time: 2023-06-22 13:32:50
This release contains 2 new features, 4 bug fixes, and 2 documentation improvements.
🆕 Features
Streaming single Document with HTTP SSE for Deployment (#5899)
In this release, we have added support for Server-Sent Events (SSE) to Jina's HTTP protocol with the Deployment orchestration. A Jina server can now stream single Documents to a client, one at a time. This is useful for applications that require a continuous stream of data, such as chatbots (using Large Language Models) or real-time translation.
Simply define an endpoint function that receives a single Document and yields Documents one by one:
from jina import Executor, requests, Document, Deployment
class MyExecutor(Executor):
@requests(on='/hello')
async def task(self, doc: Document, **kwargs):
for i in range(3):
yield Document(text=f'{doc.text} {i}')
with Deployment(
uses=MyExecutor,
port=12345,
protocol='http',
cors=True,
include_gateway=False,
) as dep:
dep.block()From the client side, you can use the new stream_doc method to receive the Documents one by one:
from jina import Client, Document
client = Client(port=12345, protocol='http', cors=True, asyncio=True)
async for doc in client.stream_doc(
on='/hello', inputs=Document(text='hello world')
):
print(doc.text)Note that the SSE client is language-independent. This feature also supports DocArray v2. For more information, see the streaming endpoints section of the Jina documentation.
Add env_from_secret option to Gateway (#5914)
We have added the env_from_secret parameter to Gateway to allow custom gateways to load secrets from Kubernetes when transformed to Kubernetes YAML in the same way as Executors.
from jina import Flow
f = Flow().config_gateway(env_from_secret={'SECRET_PASSWORD': {'name': 'mysecret', 'key': 'password'}}).add()
f.to_kubernetes_yaml()🐞 Bug Fixes
Fix error working with some data types in DocArray V2 (#5905) (#5908)
We have fixed some errors when DocArray v2 documents contained flexible dictionaries, Lists, or Tensors.
Fix reloading Executor when is loaded from config.yml (#5915)
The reload option of an Executor was not working when the Executor was loaded from config.yml, and Jina was not able to update Executor code after updates.
f = Flow().add(uses='config.yml', reload=True)
with f:
f.block()Ensure closing Executor at shutdown in Deployment with HTTP protocol (#5906)
We have fixed a bug that prevented the close method of the Executor from being executed at shutdown when the Deployment is exposed with an HTTP server.
Fix issue in mismatch endpoint when using shards (#5904)
A KeyError was raised when working with DocArray v2 in a Deployment using shards if the endpoints were not matching.
With this fix, it will properly call the default endpoint.
from jina import Deployment, Executor, requests
from docarray import BaseDoc, DocList
from docarray.typing import NdArray
fromt typing import List
class MyDoc(BaseDoc):
text: str
embedding: NdArray[128]
class MyDocWithMatchesAndScores(MyDoc):
matches: DocList[MyDoc]
scores: List[float]
class MyExec(Executor):
@requests
def foo(self, docs: DocList[MyDoc], **kwargs) -> DocList[MyDocWithMatchesAndScores]:
res = DocList[MyDocWithMatchesAndScores]()
for doc in docs:
new_doc = MyDocWithMatchesAndScores(text=doc.text, embedding=doc.embedding, matches=docs,
scores=[1.0 for _ in docs])
res.append(new_doc)
return res
d = Deployment(uses=MyExec, shards=2)
with d:
res = d.post(on='/', inputs=DocList[MyDoc]([MyDoc(text='hey ha', embedding=np.random.rand(128))]))
assert len(res) == 1📗 Documentation Improvements
🤟 Contributors
We would like to thank all contributors to this release:
- Alex Cureton-Griffiths (@alexcg1 )
- AlaeddineAbdessalem (@alaeddine-13)
- Joan Fontanals (@JoanFM )