Flyte v1.9.0 Release
In this release we're announcing two experimental features, namely (1) ArrayNode map tasks, and (2) Execution Tags.
ArrayNode map tasks
ArrayNodes are described more fully in RFC 3346, but the summary is that ArrayNode map tasks are a drop-in replacement for regular map tasks, the only difference being the submodule used to import the map_task
function.
More explicitly, let's say you have this code:
from typing import List
from flytekit import map_task, task, workflow
@task
def t(a: int) -> int:
...
@workflow
def wf(xs: List[int]) -> List[int]:
return map_task(t)(a=xs)
In order to switch to using array node map tasks you should import map_task from the flytekit.experimental
module like so:
from typing import List
from flytekit import task, workflow
from flytekit.experimental import map_task
@task
def t(a: int) -> int:
...
@workflow
def wf(xs: List[int]) -> List[int]:
return map_task(t)(a=xs)
Execution tags
Execution tags allow users to can discover their executions and other flyte entities more easily, by creating smarter groupings. The feature is described in this RFC.
As mentioned before, this feature is shipped in an experimental capacity, the idea being that we're going to incorporate the feedback of the community as we iterate. More work is expected to give prominence to the feature in flyteconsole, in the meanwhile, the feature is supported via Remote.
Flytekit
- Improve error handling in ShellTask by @pradithya in flyteorg/flytekit#1732
- use default settings for timeline deck width by @cosmicBboy in flyteorg/flytekit#1748
- Raise an exception in case of local execution of raw containers tasks by @eapolinario in flyteorg/flytekit#1745
- Update contributing.rst by @eapolinario in flyteorg/flytekit#1753
- Skip problematic pyyaml versions by @eapolinario in flyteorg/flytekit#1752
- Fail CI tests faster by @eapolinario in flyteorg/flytekit#1756
- Run unit tests on macos-latest by @eapolinario in flyteorg/flytekit#1749
- add rdzv_configs to kfpytorch elastic by @Nan2018 in flyteorg/flytekit#1751
- Fix: Disable rich logging handler when env var
FLYTE_SDK_RICH_TRACEBACKS=0
is set by @fg91 in flyteorg/flytekit#1760 - Throw warning for nested @task functions by @oliverhu in flyteorg/flytekit#1727
- Remove dependency on responses by @honnix in flyteorg/flytekit#1762
- Fix mlflow test error by @pingsutw in flyteorg/flytekit#1766
- Use phrase 'git revision SHA1' in comments and docs by @DavidMertz in flyteorg/flytekit#1761
- Dockerize docs requirements generation by @eapolinario in flyteorg/flytekit#1764
- Restrict grpcio<1.53.1 by @eapolinario in flyteorg/flytekit#1767
- Array node map task by @eapolinario in flyteorg/flytekit#1640
- Add agent ctrl-c handler to call the delete function. (Reupload) by @Future-Outlier in flyteorg/flytekit#1782
- Add tags to execution by @pingsutw in flyteorg/flytekit#1723
Flyteadmin
- Alter ID sequence to bigint by @honnix in flyteorg/flyteadmin#578
- Remove content md5 requirement by @wild-endeavor in flyteorg/flyteadmin#587
- Propagate request id on incoming and outgoing requests by @EngHabu in flyteorg/flyteadmin#582
- Update boilerplate version by @flyte-bot in flyteorg/flyteadmin#589
- Update boilerplate version by @flyte-bot in flyteorg/flyteadmin#594
- Update boilerplate version by @flyte-bot in flyteorg/flyteadmin#596
- Bumping flytestdlib by @gvashishtha in flyteorg/flyteadmin#597
- Flyte Execution tags by @pingsutw in flyteorg/flyteadmin#571
Flyteplugins
- Add support for using task execution ID fields in log URI templates by @jeevb in flyteorg/flyteplugins#372
- Fix generate check in CI by @jeevb in flyteorg/flyteplugins#377
- Remove welcomebot from boilerplate by @eapolinario in flyteorg/flyteplugins#375
- Carry over hash value for all literal types in remote caching by @nicholasjng in flyteorg/flyteplugins#378
- Send task execution metadata to out-core plugin by @honnix in flyteorg/flyteplugins#369
- Support gRPC config for agent-service plugin by @honnix in flyteorg/flyteplugins#368
- Use agent as name where it fits by @honnix in flyteorg/flyteplugins#381
- Fix deletion of elastic task resource requests by @fg91 in flyteorg/flyteplugins#379
Flytepropeller
- Update boilerplate version by @flyte-bot in flyteorg/flytepropeller#591
- fixing max parallelism by @hamersaw in flyteorg/flytepropeller#594
- Updated flytestdlib 1.0.20 by @hamersaw in flyteorg/flytepropeller#593
- Update boilerplate version by @flyte-bot in flyteorg/flytepropeller#597
- Instrument ArrayNode by @hamersaw in flyteorg/flytepropeller#550
- make singular unions castable to their underlying type by @hamersaw in flyteorg/flytepropeller#599
- correct propagation of launchplan start error by @hamersaw in flyteorg/flytepropeller#598
- Bumping flytestdlib and stow versions by @gvashishtha in flyteorg/flytepropeller#602
- Update boilerplate version by @flyte-bot in flyteorg/flytepropeller#601
- Bump flyteplugins to 1.1.15 by @eapolinario in flyteorg/flytepropeller#603
- updated flyteplugins to 1.1.16 by @hamersaw in flyteorg/flytepropeller#604
Flyteconsole
- feat: Add localStorage for selected Project/Domain by @jsonporter in flyteorg/flyteconsole#774
- Fix project selector failing test by @FrankFlitton in flyteorg/flyteconsole#780
- fix: node executions list going blank by @ursucarina in flyteorg/flyteconsole#788
- fix: launch form fixes by @ursucarina in flyteorg/flyteconsole#785
- chore: fix 404 due to bad state by @ursucarina in flyteorg/flyteconsole#797
- Fix: Launch Form fixes by @ursucarina in flyteorg/flyteconsole#798
- fix: map tasks should report caching status accurately by @ursucarina in flyteorg/flyteconsole#800
- feat: add support from structured datasets by @ursucarina in flyteorg/flyteconsole#801
- Top level masonry refactor by @FrankFlitton in flyteorg/flyteconsole#771
- fix: Tasks status out of sync by @ursucarina in flyteorg/flyteconsole#802
- chore: fix test_coverage by @ursucarina in flyteorg/flyteconsole#803
- Fix: breadcrumb feature flag priority order by @FrankFlitton in flyteorg/flyteconsole#804
- chore: fix yarn.lock by @ursucarina in flyteorg/flyteconsole#805
- fix: releases by @ursucarina in flyteorg/flyteconsole#806
- chore: fix release retry by @ursucarina in flyteorg/flyteconsole#807
- fix: specify node18, semantic release with exec and git plugins by @FrankFlitton in flyteorg/flyteconsole#808
- fix: commit linter running on generated messages by @FrankFlitton in flyteorg/flyteconsole#810
- chore: remove release git step by @FrankFlitton in flyteorg/flyteconsole#811
- fix: union value handling in launch form by @ursucarina in flyteorg/flyteconsole#812
New Contributors
- @Nan2018 made their first contribution in flyteorg/flytekit#1751
- @oliverhu made their first contribution in flyteorg/flytekit#1727
- @DavidMertz made their first contribution in flyteorg/flytekit#1761
- @Future-Outlier made their first contribution in flyteorg/flytekit#1782
- @gvashishtha made their first contribution in flyteorg/flyteadmin#597
- @nicholasjng made their first contribution in flyteorg/flyteplugins#378
- @gvashishtha made their first contribution in flyteorg/flytepropeller#602