Bugfixes
- Fixed tick display in the sensor/schedule timeline view in Dagit.
- Changed the
dagster sensor listanddagster schedule listCLI commands to include schedules and sensors that have never been turned on. - Fixed the backfill progress stats in Dagit which incorrectly capped the number of successful/failed runs.
- Improved query performance in Dagit on pipeline (or job) views, schedule views, and schedules list view by loading partition set data on demand instead of by default.
- Fixed an issue in Dagit where re-executing a pipeline that shares an identical name and graph to a pipeline in another repository could lead to the wrong pipeline being executed.
- Fixed an issue in Dagit where loading a very large DAG in the pipeline overview could sometimes lead to a render loop that repeated the same GraphQL query every few seconds, causing an endless loading state and never rendering the DAG.
- Fixed an issue with
execute_in_processwhere providing default executor config to a job would cause config errors. - Fixed an issue with default config for jobs where using an
opsconfig entry in place ofsolidswould cause a config error. - Dynamic outputs are now properly supported while using
adls2_io_manager ModeDefinitionnow validates the keys ofresource_defsat definition time.Failureexceptions no longer bypass theRetryPolicyif one is set.
Community Contributions
- Added
serviceAccount.nameto the user deployment Helm subchart and schema, thanks @jrouly!
Experimental
- To account for ECS’ eventual consistency model, the
EcsRunLauncherwill now exponentially backoff certain requests for up to a minute while waiting for ECS to reach a consistent state. - Memoization is now available from all execution entrypoints. This means that a pipeline tagged for use with memoization can be launched from dagit, the
launchCLI, and other modes of external execution, whereas before, memoization was only available viaexecute_pipelineand theexecuteCLI. - Memoization now works with root input managers. In order to use a root input manager in a pipeline that utilizes memoization, provide a string value to the
versionargument on the decorator:
from dagster import root_input_manager
@root_input_manager(version="foo")
def my_root_manager(_):
pass- The
versioned_fs_io_managernow defaults to using the storage directory of the instance as a base directory. GraphDefinition.to_jobnow accepts a tags dictionary with non-string values - which will be serialized to JSON. This makes job tags work similarly to pipeline tags and solid tags.
Documentation
- The guide for migrating to the experimental graph, job, and op APIs now includes an example of how to migrate a pipeline with a composite solid.