jina-ai/serve v3.15.0 on GitHub

Release Note (`3.15.0`)

Release time: 2023-04-14 09:55:25

This release contains 6 new features, 6 bug fixes and 5 documentation improvements.

🆕 Features

HTTP and composite protocols for Deployment (#5764)

When using a Deployment to serve a single Executor, you can now expose it via HTTP or a combination of HTTP and gRPC protocols:

from jina import Deployment, Executor, requests

class MyExec(Executor):

    @requests(on='/bar')
    def bar(self, docs, **kwargs):
        pass

dep = Deployment(protocol=['http', 'grpc'], port=[12345, 12346], uses=MyExec)

with dep:
    dep.block()

With this, you can also access the OpenAPI schema in localhost:12345/docs:

Force network mode option (#5789)

When using a containerized Executor inside a Deployment or as part of a Flow, under some circumstances you may want to force the network mode to make sure the container is reachable by the Flow or Deployment to ensure readiness. This ensures that the Docker Python SDK runs the container with the relevant options.

For this, we have added the argument force_network_mode to enable this.

You can set this argument to any of these options:

AUTO: Automatically detect the Docker network.
HOST: Use the host network.
BRIDGE: Use a user-defined bridge network.
NONE: Use None as the network.

from jina import Deployment

dep = Deployment(uses='jinaai+docker://TransformerTorchEncoder', force_network_mode='None')
with dep:
    dep.block()

Allow disabling thread lock (#5771)

When an Executor exposes a synchronous method (not a coroutine) and exposes this method via the @requests decorator, Jina makes sure that each request received is run in a thread.

This thread is however locked with a threading.Lock object to protect the user from potential hazards of multithreading while leaving the Executor free to respond to health checks coming from the outside or from orchestrator frameworks such as Kubernetes. This lock can be bypassed if the allow_concurrent argument is passed to the Executor.

from jina import Deployment, Executor, requests

class MyExec(Executor):

    @requests(on='/bar')
    def bar(self, docs, **kwargs):
        pass

dep = Deployment(allow_concurrent=True, uses=MyExec)

with dep:
    dep.block()

`grpc_channel_options` for custom gRPC options for the channel (#5765)

You can now pass grpc_channel_options to allow granular tuning of the gRPC connectivity from the Client or Gateway. You can check the options in gRPC Python documentation

client = Client(grpc_channel_options={'grpc.max_send_message_length': -1})

Create Deployments from the CLI (#5756)

New you can create from the Jina CLI to create a first project to deploy a single Deployment in the same way it was possible to create one for a Flow.

Now the jina new command accepts a new type argument that can be flow or deployment.

jina new hello-world --type flow
jina new hello-world --type deployment

Add `replicas` argument to Gateway for Kubernetes (#5711)

To scale the Gateway in Kubernetes or in JCloud, you can now add the replicas arguments to the gateway.

from jina import Flow
f = Flow().config_gateway(replicas=3).add()
f.to_kubernetes_yaml('./k8s_yaml_path')

jtype: Flow
version: '1'
with: {}
gateway:
  replicas: 3
executors:
- name: executor0

🐞 Bug Fixes

Retry client gRPC stream and unary RPC methods (#5733)

The retry mechanism parameters were not properly respected by the Client in prior releases. This is now fixed and will improve the robustness against transient errors.

from jina import Client, DocumentArray

Client(host='...').post(
    on='/',
    inputs=DocumentArray.empty(),
    max_attempts=100,
)

Allow HTTP timeout (#5797)

When using the Client to send data to an HTTP service, the connection timed out after five minutes (the default setting for aiohttp). This can now be edited for cases where a request may take longer, thus avoiding the Client disconnecting after a longer period.

from jina import Client, DocumentArray

Client(protocol='http').post(
    on='/',
    inputs=DocumentArray.empty(),
    timeout=600,
)

Enable root logging at all times (#5736)

The JINA_LOG_LEVEL environment variable controls the log level of the JinaLogger. Previously the debug logging of other dependencies was not respected. Now they can be enabled.

logging.get_logger('urllib3').setLevel(logging.DEBUG)

Fix Gateway tensor serialization (#5752)

In prior releases, when an HTTP Gateway was run without torch installed and connected to an Executor returning torch.Tensor as part of the Documents, the Gateway couldn't serialize the Documents back to the Client, leading to a no module torch error. This is now fixed and works without installing torch in the Gateway container or system.

from jina import Flow, Executor, Document, DocumentArray, requests

import torch

class DummyTorchExecutor(Executor):
    @requests
    def foo(self, docs: DocumentArray, **kwargs):
        for d in docs:
            d.embedding = torch.rand(1000)
            d.tensor = torch.rand(1000)

from jina import Flow, Executor, Document, DocumentArray, requests

flow = Flow().config_gateway(port=12346, protocol='http').add(port='12345', external=True)

with flow:
    docs = flow.post(on='/', inputs=Document())
    print(docs[0].embedding.shape)
    print(docs[0].tensor.shape)

Composite Gateway tracing (#5741)

Previously, tracing didn't work for Gateways that exposed multiple protocols:

from jina import Flow

f = Flow(port=[12345, 12346], protocol=['http', 'grpc'], tracing=True).add()
with f:
   f.block()

Adapt to DocArray v2 (#5742)

Jina depends on DocArray's data structures. This version adds support for DocArray v2's upcoming major changes.

The involves naming conventions:

DocumentArray ➡️ DocList
BaseDocument ➡️ BaseDoc

from jina import Deployment, Executor, requests
from docarray import DocList, BaseDoc
from docarray.documents import ImageDoc
from docarray.typing import AnyTensor

import numpy as np

class InputDoc(BaseDoc):
    img: ImageDoc

class OutputDoc(BaseDoc):
    embedding: AnyTensor

class MyExec(Executor):
    @requests(on='/bar')
    def bar(
        self, docs: DocList[InputDoc], **kwargs
    ) -> DocumentArray[OutputDoc]:
        docs_return = DocList[OutputDoc](
            [OutputDoc(embedding=np.zeros((100, 1))) for _ in range(len(docs))]
        )
        return docs_return

with Deployment(uses=MyExec) as dep:
    docs = dep.post(
        on='/bar',
        inputs=InputDoc(img=ImageDoc(tensor=np.zeros((3, 224, 224)))),
        return_type=DocList[OutputDoc],
    )
    assert docs[0].embedding.shape == (100, 1)
    assert docs.__class__.document_type == OutputDoc

📗 Documentation improvements

JCloud Flow name customization (#5778)
JCloud docs revamp for instance (#5759)
Fix Colab link (#5760)
Remove docsQA (#5743)
Misc polishing

🤟 Contributors

We would like to thank all contributors to this release:

Girish Chandrashekar (@girishc13)
Asuzu Kosisochukwu (@asuzukosi)
AlaeddineAbdessalem (@alaeddine-13)
Zac Li (@zac-li)
nikitashrivastava29 (@nikitashrivastava29)
samsja (@samsja)
Alex Cureton-Griffiths (@alexcg1)
Joan Fontanals (@JoanFM)
Deepankar Mahapatro (@deepankarm)

jina-ai/serve v3.15.0 💫 Release v3.15.0 on GitHub

Release Note (3.15.0)