New
- Updated dagstermill to better support job/op/graph changes by adding a
define_dagstermill_op
factory function. Also updated documentation and examples to reflect these changes. - Changed run history for jobs in Dagit to include legacy mode tags for runs that were created from pipelines that have since been converted to use jobs.
- The new get_dagster_logger() method is now importable from the top level dagster module (
from dagster import get_dagster_logger
) - [dagster-dbt] All dagster-dbt resources (
dbt_cli_resource
,dbt_rpc_resource
, anddbt_rpc_sync_resource
) now support thedbt ls
command:context.resources.dbt.ls()
. - Added
ins
andouts
properties toOpDefinition
. - Updated the run status favicon of the Run page in Dagit.
- There is now a
resources_config
argument onbuild_solid_context
. The config argument has been renamed tosolid_config
. - [helm] When deploying Redis using the Dagster helm chart, by default the new cluster will not require authentication to start a connection to it.
- [dagster-k8s] The component name on Kubernetes jobs for run and step workers is now
run_worker
andstep_worker
, respectively. - Improved performance for rendering the Gantt chart on the Run page for runs with very long event logs.
Bugfixes
- Fixed a bug where decorating a job with a hook would create a pipeline.
- Fixed a bug where providing default logger config to a job would break with a confusing error.
- Fixed a bug with retrieving output results from a mapped input on
execute_in_process
- Fixed a bug where schedules referencing a job were not creating runs using that job’s default run config.
- [dagster-k8s] Fixed a bug where the retry mode was not being passed along through the k8s executor.
Breaking Changes
- The first argument on
Executor.execute(...)
has changed frompipeline_context
toplan_context
Community Contributions
- When using multiple Celery workers in the Dagster helm chart, each worker can now be individually configured. See the helm chart for more information. Thanks @acrulopez!
- [dagster-k8s] Changed Kubernetes job containers to use the fixed name
dagster
, rather than repeating the job name. Thanks @skirino!
Experimental
-
[dagster-docker] Added a new
docker_executor
which executes steps in separate Docker containers. -
The dagster-daemon process can now detect hanging runs and restart crashed run workers. Currently
only supported for jobs using thedocker_executor
andk8s_job_executor
. Enable this feature in your dagster.yaml with:run_monitoring: enabled: true
Documentation coming soon. Reach out in the #dagster-support Slack channel if you are interested in using this feature.
Documentation
- Adding “Python Logging” back to the navigation pane.
- Updated documentation for
dagster-aws
,dagster-github
, anddagster-slack
to reference job/op/graph APIs.