astronomer/astronomer-cosmos astronomer-cosmos-v1.16.0rc2

v1.16.0rc2

on GitHub

Pre-release candidate of Cosmos 1.16.0.

Second release candidate. It adds 14 milestone PRs merged since 1.16.0rc1: more dbt 2 (dbt-oss) fixes, the new minimum Python (3.11) and Apache Airflow (2.11) versions, and docs. The head branch was synced with main so this cut contains all of them. Please try it and report any problems before the 1.16.0 GA. See the release PR for full provenance, the complete 178-PR changelog, and the test plan: #3073.

Breaking Changes

• Raised the minimum supported Python version to 3.11. Python 3.10 has reached End of Life per the Python release schedule. Users on Python 3.10 can stay on Cosmos 1.15.x or move to Python 3.11 or newer. See #3159.
• Raised the minimum supported Apache Airflow version to 2.11. Apache Airflow 2.9 and 2.10 (Astro Runtime 11 and 12) are past End of Maintenance per the Astronomer Runtime lifecycle policy and are no longer tested or supported. Users who cannot upgrade Airflow should stay on Cosmos 1.15.x. See #3152.

Behaviour Changes

• Like dbt, a bare selector (a folder, a package or a name, for example staging or the project name) and fqn: no longer select sources under LoadMode.DBT_MANIFEST. Before, a source matched when the folder of its YAML file or its package matched, so with source rendering on, a bare folder or package selector also rendered the source and the models that only read it. Use source: to select a source; package: and path: still select sources. See #3147.

Enhancements

• Drop support for Python 3.10 by @pankajastro in #3159
• Drop support for Apache Airflow 2.9 and 2.10 by @jedcunningham in #3152

Bug Fixes

• Do not select sources with bare and fqn: selectors under LoadMode.DBT_MANIFEST, as dbt does by @pankajastro in #3147
• Skip the --no-static-parser flag under InvocationMode.DBT_RUNNER when dbt 2 (dbt-oss) is installed. dbt 2 has no static parser, so the flag logged NoLongerSupportedOption (dbt1700) on every command, which failed the task when warn_error=True by @pankajastro in #3144
• Find the compiled SQL of dbt 2 (dbt-oss) models in the ExecutionMode.WATCHER producer. dbt 2 reports node_path relative to the project instead of the models folder, so the compiled SQL was never found by @pankajastro in #3145
• In InvocationMode.SUBPROCESS, read the source freshness warnings that dbt 2 (dbt-oss) prints as Warned source <name> (...), so DbtSourceLocalOperator calls on_warning_callback with the source in test_names and the line dbt printed in test_results by @pankajastro in #3146
• Detect dbt 2 (dbt-oss) source freshness warnings in the Kubernetes and GKE operators, so on_warning_callback runs for a stale source by @pankajastro in #3156
• In InvocationMode.SUBPROCESS, list the [error] [<Name> (dbt<code>)] blocks of failed models, macros and stale sources of dbt 2 (dbt-oss) in the exception of a failed command by @pankajastro in #3148
• In InvocationMode.DBT_RUNNER, report a dbt 2 (dbt-oss) failure that is not tied to one node, such as a failing run-operation macro, in one line instead of listing every node of the project by @pankajastro in #3149
• Serve the assets/ and info_schema/ files of the dbt 2 (dbt-oss) docs site in the Airflow 2 plugin, which loaded as a blank page before by @pankajastro in #3155
• Serve the assets/ and info_schema/ files of the dbt 2 (dbt-oss) docs site in the Airflow 3 plugin, which loaded as a blank page before by @pankajastro in #3154

Docs

• Align Runtime support in the compatibility policy with End of Maintenance by @pankajkoti in #3150
• Document dbt OSS (dbt 2) in Kubernetes Pods by @pankajastro in #3157
• Fix the enable_dataset_alias value that Airflow 3 rejects in the scheduling guide by @jedcunningham in #3153

Milestone: Cosmos 1.16.0
Full changelog: astronomer-cosmos-v1.16.0rc1...astronomer-cosmos-v1.16.0rc2