New
- Dagster code servers now wait to shut down until any calls that they are running have finished, preventing them from stopping while in the middle of executing sensor ticks or other long-running operations.
- The
dagster execute job
cli now accepts—-op-selection
(thanks @silent-lad!) - [ui] Option (Alt) + R now reloads all code locations (OSS only)
Bugfixes
- Adds a check to validate partition mappings when directly constructing
AssetsDefinition
instances. - Assets invoked in composition functions like
@graph
and@job
now work again, fixing a regression introduced in 1.4.5. - Fixed an issue where a race condition with parallel runs materializing the same asset could cause a run to hang.
- Fixed an issue where including a resource in both a schedule and a job raised a “Cannot specify resource requirements” exception when the definitions were loaded.
- The
ins
argument tograph_asset
is now respected correctly. - Fixed an issue where the daemon process could sometimes stop with a heartbeat failure when the first sensor it ran took a long time to execute.
- Fixed an issue where
dagster dev
failed on startup when theDAGSTER_GRPC_PORT
`environment variable was set in the environment. deps
arguments for an asset can now be specified as an iterable instead of a sequence, allowing for sets to be passed.- [dagster-aws] Fixed a bug where the S3PickleIOManager didn’t correctly handle missing partitions when allow_missing_partitions was set. Thanks @o-sirawat!
- [dagster-k8s] in the helm chart, the daemon
securityContext
setting now applies correctly to all init containers (thanks @maowerner!)
Community Contributions
- [dagster-databricks] Migrated to use new official databricks Python SDK. Thanks @judahrand!
Experimental
- New APIs for defining and executing checks on software-defined assets. These APIs are very early and subject to change. The corresponding UI has limited functionality. Docs
- Adds a new auto-materialize skip rule
AutoMaterializeRule.skip_on_not_all_parents_updated
that enforces that an asset can only be materialized if all parents have been materialized since the asset's last materialization. - Exposed an auto-materialize skip rule –
AutoMaterializeRule.skip_on_parent_missing
–which is already part of the behavior of the default auto-materialize policy. - Auto-materialize evaluation history will now be stored for 1 month, instead of 1 week.
- The auto-materialize asset daemon now includes more logs about what it’s doing for each asset in each tick in the Dagster Daemon process output.
Documentation
- [dagster-dbt] Added reference docs for
dagster-dbt project scaffold
.
Dagster Cloud
- Fixed an issue where the Docker agent would sometimes fail to load code locations with long names with a hostname connection error.