github astronomer/astronomer-cosmos astronomer-cosmos-v1.14.2
v1.14.2

latest releases: astronomer-cosmos-v1.15.1, astronomer-cosmos-v1.15.1a5, astronomer-cosmos-v1.15.1a4...
3 months ago

Behaviour Changes

These changes adjust observable behaviour of the ExecutionMode.WATCHER execution mode.
None of them break the public Cosmos API, but users relying on undocumented internals
(graph wiring assertions, XCom backup Variable names, retry-on-recovery semantics, or
retry log format) should review before upgrading.

  • ExecutionMode.WATCHER + depends_on_past=True: when the producer task has
    depends_on_past=True (typically set via default_args), the producer-done gateway
    task inside DbtTaskGroup is now wired downstream of every consumer task, in addition
    to the producer. This is required so that wait_for_downstream gating behaves
    correctly across DAG runs and the task group acts as a single unit that must fully
    succeed before the next run starts. Users with depends_on_past=False (the default)
    see no topology change. See #2615.
  • ExecutionMode.WATCHER downstream retry on upstream recovery: dbt models that were
    skipped after an upstream-failure event are now retried in the same DAG run when the
    upstream task succeeds on retry. Previously these models remained skipped for the run.
    See #2684.
  • ExecutionMode.WATCHER consumer-retry log format: the consumer's fallback dbt
    invocation no longer inherits the producer's internal --log-format json flag, so
    retry task logs now default to dbt's normal text format. Users who relied on JSON output
    in retry logs can opt in via operator_args={"dbt_cmd_flags": ["--log-format", "json"]}.
    See #2713.
  • ExecutionMode.WATCHER XCom-backup Variable key scheme: the per-model XCom backup
    Variable key now includes the full task-group path and sanitises disallowed characters
    (+ / :) from run_id. External monitoring or cleanup scripts that match the
    old key pattern will need updating. See #2629 and #2683.

Bug Fixes

  • Sanitize disallowed characters from XCom backup variable key by @MichaelRBlack in #2629
  • Prevent watcher producers from colliding on one XCom-backup key by @tatiana in #2683
  • Retry watcher downstream models on upstream-failure recovery by @tatiana in #2684
  • Fix ExecutionMode.WATCHER interaction with depends_on_past by @johnhoran in #2615
  • Strip --log-format from producer flags on watcher consumer retry by @tatiana in #2713
  • Fix duplicate deferrable kwarg in DbtRunAirflowAsyncBigqueryOperator by @pankajastro in #2616
  • Fix dbt docs iframe src missing deployment path prefix by @pankajastro in #2640
  • Defer TaskInstance import in cluster policy to fix Sentry init crash by @pankajastro in #2662
  • Restore type hints broken by lazy imports in cosmos/__init__.py by @pankajastro in #2647
  • Fix ExecutionMode.WATCHER non-dbt stdout being suppressed from logs by @pankajastro in #2654
  • Fix test sensor retry behaviour in ExecutionMode.WATCHER by @pankajkoti in #2658
  • Fix watcher fallback selector for versioned dbt models by @pankajkoti in #2659
  • Break out of iframe from Airflow 2 dbt Docs 404 link by @pankajastro in #2685

Docs

Others

Don't miss a new astronomer-cosmos release

NewReleases is sending notifications on new releases.