pypi dagster-slack 0.12.7

latest releases: 0.14.3, 0.14.3rc0, 0.14.2...
3 years ago

New

  • In Dagit, the repository locations list has been moved from the Instance Status page to the Workspace page. When repository location errors are present, a warning icon will appear next to “Workspace” in the left navigation.
  • Calls to context.log.info() and other similar functions now fully respect the python logging API. Concretely, log statements of the form context.log.error(“something %s happened!”, “bad”) will now work as expected, and you are allowed to add things to the “extra” field to be consumed by downstream loggers: context.log.info("foo", extra={"some":"metadata"}).
  • Utility functions config_from_files, config_from_pkg_resources, and config_from_yaml_strings have been added for constructing run config from yaml files and strings.
  • DockerRunLauncher can now be configured to launch runs that are connected to more than one network, by configuring the networks key.

Bugfixes

  • Fixed an issue with the pipeline and solid Kubernetes configuration tags. env_from and volume_mounts are now properly applied to the corresponding Kubernetes run worker and job pods.
  • Fixed an issue where Dagit sometimes couldn’t start up when using MySQL storage.
  • [dagster-mlflow] The end_mlflow_run_on_pipeline_finished hook now no longer errors whenever invoked.

Breaking Changes

  • Non-standard keyword arguments to context.log calls are now not allowed. context.log.info("msg", foo="hi") should be rewritten as context.log.info("msg", extra={"foo":"hi"}).
  • [dagstermill] When writing output notebook fails, e.g. no file manager provided, it won't yield AssetMaterialization. Previously, it would still yield an AssetMaterialization where the path is a temp file path that won't exist after the notebook execution.

Experimental

  • Previously, in order to use memoization, it was necessary to provide a resource version for every resource used in a pipeline. Now, resource versions are optional, and memoization can be used without providing them.
  • InputContext and OutputContext now each has an asset_key that returns the asset key that was provided to the corresponding InputDefinition or OutputDefinition.

Documentation

  • The Spark documentation now discusses all the ways of using Dagster with Spark, not just using PySpark

Don't miss a new dagster-slack release

NewReleases is sending notifications on new releases.