github dagster-io/dagster 0.13.5

latest releases: 1.9.2, dagster-sigma/v0.25.2rc3, dagster-sigma/v0.25.2rc2...
3 years ago

New

  • [dagster-fivetran] A new dagster-fivetran integration allows you to launch Fivetran syncs and monitor their progress from within Dagster. It provides a pre-built fivetran_sync_op, as well as a more flexible fivetran_resource for more customized use cases. Check out the api docs to learn more!
  • When inferring a graph/job/op/solid/pipeline description from the docstring of the decorated function, we now dedent the docstring even if the first line isn’t indented. This allows descriptions to be formatted nicely even when the first line is on the same line as the triple-quotes.
  • The SourceHashVersionStrategy class has been added, which versions op and resource code. It can be provided to a job like so:
from dagster import job, SourceHashVersionStrategy

@job(version_strategy=SourceHashVersionStrategy())
def my_job():
     ...
  • [dagit] Improved performance on the initial page load of the Run page, as well as the partitions UI / launch backfill modal
  • [dagit] Fixed a bug where top-level graphs in the repo could not be viewed in the Workspace > Graph view.

Bugfixes

  • Fixed an issue where turning a partitioned schedule off and on again would sometimes result in unexpected past runs being created. (#5604)
  • Fixed an issue where partition sets that didn’t return a new copy of run configuration on each function call would sometimes apply the wrong config to partitions during backfills.
  • Fixed rare issue where using dynamic outputs in combination with optional outputs would cause errors when using certain executors.
  • [dagster-celery-k8s] Fixed bug where CeleryK8s executor would not respect job run config
  • [dagit] Fixed bug where graphs would sometimes appear off-center.

Breaking Changes

  • In 0.13.0, job CLI commands executed via dagster job selected both pipelines and jobs. This release changes the dagster job command to select only jobs and not pipelines.

Community Contributions

  • [dagster-dask] Updated DaskClusterTypes to have the correct import paths for certain cluster managers (thanks @kudryk!)
  • [dagster-azure] Updated version requirements for Azure to be more recent and more permissive (thanks @roeap !)
  • [dagster-shell] Ops will now copy the host environment variables at runtime, rather than copying them from the environment that their job is launched from (thanks @alexismanuel !)

Documentation

  • The job, op, graph migration guide was erroneously marked experimental. This has been fixed.

Don't miss a new dagster release

NewReleases is sending notifications on new releases.