jina-ai/serve v3.16.0

💫 Release v3.16.0

on GitHub

Release Note (3.16.0)

Release time: 2023-05-05 06:35:58

This release contains 4 new features, 5 bug fixes and 8 documentation improvements.

🆕 Features

(Beta) Replicate with consensus stateful Executors using RAFT algorithm with new DocArray version (docarray >= 0.30) (#5564)

When scaling Executors inside a Deployment, you can now ensure internal state (if the Executor has one) can be synced across every replica by ensuring they all work in consensus. This means the internal state of every replica will be consistent and they can thus serve requests in an equivalent manner.

For this, you need to decorate the Executor methods that alter its inner state with the @write decorator. Then, when adding the Executor inside a Deployment, you need to add the stateful=True flag and optionally configure the ports of every peer in the replication cluster using the --peer-ports argument:

from jina import Deployment, Executor, requests
from jina.serve.executors.decorators import write
from docarray import DocList
from docarray.documents import TextDoc

class MyStateStatefulExecutor(Executor):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self._docs_dict = {}
     
    @write
    @requests(on=['/index'])
    def index(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        for doc in docs:
            self._docs_dict[doc.id] = doc
    
    @requests(on=['/search'])
    def search(self,  docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        for doc in docs:
            self.logger.debug(f'Searching against {len(self._docs_dict)} documents')
            doc.text = self._docs_dict[doc.id].text

d = Deployment(name='stateful_executor', 
               uses=MyStateStatefulExecutor,
               replicas=3, 
               stateful=True, 
               peer_ports=[12345, 12346, 12347])
with d:
    d.block()

The consensus module in Jina will ensure that the three replicas all hold the same state. Jina uses the RAFT algorithm (https://raft.github.io/) to provide this feature.

Support HTTP and combined protocols for Deployment with new DocArray version (docarray >= 0.30) (#5826)

You can now use the new DocArray version when using HTTP as protocol to serve a Deployment, or a composition of HTTP and gRPC.

This allows OpenAPI specs matching the exact Document schemas defined by the Executor:

from jina import Executor, requests
from docarray import DocList, BaseDoc
from docarray.documents import ImageDoc
from docarray.typing import AnyTensor

import numpy as np

class InputDoc(BaseDoc):
    img: ImageDoc

class OutputDoc(BaseDoc):
    embedding: AnyTensor

class MyExec(Executor):
    @requests(on='/bar')
    def bar(
            self, docs: DocList[InputDoc], **kwargs
    ) -> DocList[OutputDoc]:
        docs_return = DocList[OutputDoc](
            [OutputDoc(embedding=np.zeros((100, 1))) for _ in range(len(docs))]
        )
        return docs_return

d = Deployment(uses=MyExec, protocol='http')

with d:
    d.block()

Support sharding in Deployment with new DocArray version (docarray >= 0.30) (#5828)

Using shards inside a Deployment (where the Executor works with docarray>=0.30) will now work the same as with previous versions of docarray as described in the documentation.

Clickable link to /docs OpenAPI endpoint (#5854)

When starting a Flow or a Deployment with the HTTP protocol, the /docs and /redocs links will now appear complete and can be clicked to open the browser directly from the terminal.

Previously, you would see:

Now:

🐞 Bug Fixes

Improved platform-specific dependency for uvloop. (#5841)

Fix installation issues when downstream dependencies try to install Jina on Windows using poetry.

Use utf-8 encoding when opening files (#5821)

Use utf-8 encoding when opening files to fix potential problems when using Jina in Windows.

Fix new Deployment CLI (#5819)

Fix the output YAML file when using jina new <project-name> --type deployment. It previously dumped an invalid Deployment YAML. The syntax has now been fixed so that it can be deployed.

Use container stop so that containers running cuda can be killed (#5816)

Use stop instead of kill from the Python Docker SDK. This allows containerized Executors that are run with conda run to terminate, since conda does not propagate its captured signals to its Python subprocesses.

Catch ContentTypeError for retry in post requests (#5825)

ContentTypeError responses in the Client will now be caught and retries applied if required.

📗 Documentation Improvements

• Update Slack count (#5842)
• Add storage section (#5831)
• Remove layer from orchestration layer title (#5829)
• Fix broken links (#5822)
• Add docs for jc recreate (#5815)
• Point to legacy DocumentArray docs (#5810)
• Fix some English in docstrings (#5718)
• Fix some links in scalability chapter (#5809)

🤟 Contributors

We would like to thank all contributors to this release:

• notandor (@NotAndOr )
• Deepankar Mahapatro (@deepankarm )
• Alex Cureton-Griffiths (@alexcg1 )
• Tanguy Abel (@Tanguyabel )
• Nikolas Pitsillos (@npitsillos )
• Felix Mönckemeyer (@NemesisFLX)
• Joan Fontanals (@JoanFM)
• Florian Hönicke (@florian-hoenicke )