dagster-io/dagster 0.14.14

on GitHub

New

• When viewing a config schema in the Dagit launchpad, default values are now shown. Hover over an underlined key in the schema view to see the default value for that key.

• dagster, dagit, and all extension libraries (dagster-*) now contain py.typed files. This exposes them as typed libraries to static type checking tools like mypy. If your project is using mypy or another type checker, this may surface new type errors. For mypy, to restore the previous state and treat dagster or an extension library as untyped (i.e. ignore Dagster’s type annotations), add the following to your configuration file:

  [mypy-dagster]  (or e.g. mypy-dagster-dbt)
  follow_imports = "skip"
  

• Op retries now surface the underlying exception in Dagit.

• Made some internal changes to how we store schema migrations across our different storage implementations.

• build_output_context now accepts an asset_key argument.

• They key argument to the SourceAsset constructor now accepts values that are strings or sequences of strings and coerces them to AssetKeys.

• You can now use the + operator to add two AssetGroups together, which forms an AssetGroup that contains a union of the assets in the operands.

• AssetGroup.from_package_module, from_modules, from_package_name, and from_current_module now accept an extra_source_assets argument that includes a set of source assets into the group in addition to the source assets scraped from modules.

• AssetsDefinition and AssetGroup now both expose a to_source_assets method that return SourceAsset versions of their assets, which can be used as source assets for downstream AssetGroups.

• Repositories can now include multiple AssetGroups.

• The new prefixed method on AssetGroup returns a new AssetGroup where a given prefix is prepended to the asset key of every asset in the group.

• Dagster now has a BoolMetadataValue representing boolean-type metadata. Specifying True or False values in metadata will automatically be casted to the boolean type.

• Tags on schedules can now be expressed as nested JSON dictionaries, instead of requiring that all tag values are strings.

• If an exception is raised during an op, Dagster will now always run the failure hooks for that op. Before, certain system exceptions would prevent failure hooks from being run.

• mapping_key can now be provided as an argument to build_op_context/build_solid_context. Doing so will allow the use of OpExecutionContext.get_mapping_key().

Bugfixes

• [dagit] Previously, when viewing a list of an asset’s materializations from a specified date/time, a banner would always indicate that it was a historical view. This banner is no longer shown when viewing the most recent materialization.
• [dagit] Special cron strings like @daily were treated as invalid when converting to human-readable strings. These are now handled correctly.
• The selection argument to AssetGroup.build_job now uses > instead of . for delimiting the components within asset keys, which is consistent with how selection works in Dagit.
• [postgres] passwords and usernames are now correctly url quoted when forming a connection string. Previously spaces were replaced with +.
• Fixed an issue where the celery_docker_executor would sometimes fail to execute with a JSON deserialization error when using Dagster resources that write to stdout.
• [dagster-k8s] Fixed an issue where the Helm chart failed to work when the user code deployment subchart was used in a different namespace than the main dagster Helm chart, due to missing configmaps.
• [dagster-airbyte] When a Dagster run is terminated while executing an Airbyte sync operation, the corresponding Airbyte sync will also be terminated.
• [dagster-dbt] Log output from dbt cli commands will no longer have distracting color-formatting characters.
• [dagit] Fixed issue where multi_assets would not show correct asset dependency information.
• Fixed an issue with the sensor daemon, where the sensor would sometimes enter a race condition and overwrite the sensor status.

Community Contributions

• [dagster-graphql] The Python DagsterGraphQLClient now supports terminating in-progress runs using client.terminate_run(run_id). Thanks @Javier162380!

Experimental

• Added an experimental view of the Partitions page / Backfill page, gated behind a feature flag in Dagit.