[Beta] v0.16.0b16

This is hopefully the last beta release. We'll let this sit for a week and then cut a formal release. Any bug fixes done next week will likely just go into that release unless it's urgent.

With this release...

Map Task

We've added a map task. Map tasks are similar to the map functionality you find in Python and apply a list over an existing task.

@workflow
def my_wf(a: typing.List[int]) -> int:
    x = map_task(t1, metadata=TaskMetadata(retries=1))(a=a)
    return t2(x=x)

Please see the PR and code comments for more information. Further documentation is forthcoming.

Note that map tasks are different (more optimized) than dynamic tasks because if you map a task over a thousand elements it still remains just one task. A dynamic task would create a thousand nodes.

Task Resolver

Shoring up the execution side of flytekit (when the Flyte backend actually runs the task container on K8s) has been a long time coming. Again please see notes in the PR and code comments for more information, but this updates the way tasks are loaded at execution time and adds flexibility.

Previously, tasks were always loaded by one call to Python's importlib.import_module(). Now, at serialization time, the 'task resolver', a default for which is provided, constructs arguments that, at execution time, the same resolver uses to resurrect/rehydrate the task. The default resolver merely does the same thing as previous, calling import_module and looking up the task name.

Misc

• Fixed an issue where subworkflows called in conditionals weren't getting included to the workflow closure.
• Fixed an issue where subworkflows were running even in false conditional branches during local execution
• Added custom Protobuf Type support
• Fixed the ability to construct launch plan from a ReferenceWorkflow