apache/airflow 1.10.11

Airflow 1.10.11, 2020-07-10

on GitHub

New Features

• Add task instance mutation hook (#8852)
• Allow changing Task States Colors (#9520)
• Add support for AWS Secrets Manager as Secrets Backend (#8186)
• Add airflow info command to the CLI (#8704)
• Add Local Filesystem Secret Backend (#8596)
• Add Airflow config CLI command (#8694)
• Add Support for Python 3.8 (#8836)(#8823)
• Allow K8S worker pod to be configured from JSON/YAML file (#6230)
• Add quarterly to crontab presets (#6873)
• Add support for ephemeral storage on KubernetesPodOperator (#6337)
• Add AirflowFailException to fail without any retry (#7133)
• Add SQL Branch Operator (#8942)

Bug Fixes

• Use NULL as dag.description default value (#7593)
• BugFix: DAG trigger via UI error in RBAC UI (#8411)
• Fix logging issue when running tasks (#9363)
• Fix JSON encoding error in DockerOperator (#8287)
• Fix alembic crash due to typing import (#6547)
• Correctly restore upstream_task_ids when deserializing Operators (#8775)
• Correctly store non-default Nones in serialized tasks/dags (#8772)
• Correctly deserialize dagrun_timeout field on DAGs (#8735)
• Fix tree view if config contains " (#9250)
• Fix Dag Run UI execution date with timezone cannot be saved issue (#8902)
• Fix Migration for MSSQL (#8385)
• RBAC ui: Fix missing Y-axis labels with units in plots (#8252)
• RBAC ui: Fix missing task runs being rendered as circles instead (#8253)
• Fix: DagRuns page renders the state column with artifacts in old UI (#9612)
• Fix task and dag stats on home page (#8865)
• Fix the trigger_dag api in the case of nested subdags (#8081)
• UX Fix: Prevent undesired text selection with DAG title selection in Chrome (#8912)
• Fix connection add/edit for spark (#8685)
• Fix retries causing constraint violation on MySQL with DAG Serialization (#9336)
• [AIRFLOW-4472] Use json.dumps/loads for templating lineage data (#5253)
• Restrict google-cloud-texttospeach to <v2 (#9137)
• Fix pickling failure when spawning processes (#8671)
• Pin Version of Azure Cosmos to <4 (#8956)
• Azure storage 0.37.0 is not installable any more (#8833)
• Fix modal_backdrop z-index in the UI (#7313)
• Fix Extra Links in Gannt View (#8308)
• Bug fix for EmrAddStepOperator init with cluster_name error (#9235)
• Fix KubernetesPodOperator pod name length validation (#8829)
• Fix non updating DAG code by checking against last modification time (#8266)
• BugFix: Unpausing a DAG with catchup=False creates an extra DAG run (#8776)

Improvements

• Improve add_dag_code_table migration (#8176)
• Persistent display/filtering of DAG status (#8106)
• Set unique logger names (#7330)
• Update the version of cattrs from 0.9 to 1.0 to support Python 3.8 (#7100)
• Reduce response payload size of /dag_stats and /task_stats (#8655)
• Add authenticator parameter to snowflake_hook (#8642)
• Show "Task Reschedule" table in Airflow Webserver (#9521)
• Change worker_refresh_interval fallback to default of 30 (#9588)
• Use pfromat instead of str to render arguments in WebUI (#9587)
• Simplify DagFileProcessorManager (#7521)
• Reload gunicorn when plugins has beeen changed (#8997)
• Fix the default value for store_dag_code (#9554)
• Add support for fetching logs from running pods (#8626)
• Persist start/end date and duration for DummyOperator Task Instance (#8663)
• Ensure "started"/"ended" in tooltips are not shown if job not started (#8667)
• Add context to execution_date_fn in ExternalTaskSensor (#8702)
• Avoid color info in response of /dag_stats & /task_stats (#8742)
• Make loading plugins from entrypoint fault-tolerant (#8732)
• Refactor Kubernetes worker config (#7114)
• Add default conf parameter to Spark JDBC Hook (#8787)
• Allow passing backend_kwargs to AWS SSM client (#8802)
• Filter dags by clicking on tag (#8897)
• Support k8s auth method in Vault Secrets provider (#8640)
• Monitor pods by labels instead of names (#6377)
• Optimize count query on /home (#8729)
• Fix json string escape in tree view (#8551)
• Add TaskInstance state to TI Tooltip to be colour-blind friendlier (#8910)
• Add a tip to trigger DAG screen (#9049)
• Use Markup for htmlcontent for landing_times (#9242)
• Pinning max pandas version to 2.0 (lesser than) to allow pandas 1.0 (#7954)
• Update example webserver_config.py to show correct CSRF config (#8944)
• Fix displaying Executor Class Name in "Base Job" table (#8679)
• Use existing DagBag for 'dag_details' & 'trigger' Endpoints (#8501)
• Flush pending Sentry exceptions before exiting (#7232)
• Display DAG run conf in the list view (#6794)
• Fix performance degradation when updating dagrun state (#8435)
• Don't use the |safe filter in code, it's risky (#9180)
• Validate only task commands are run by executors (#9178)
• Show Deprecation warning on duplicate Task ids (#8728)
• Move DAG._schedule_interval logic out of DAG.__init__ (#8225)
• Make retrieving Paused Dag ids a separate method (#7587)
• Bulk fetch paused_dag_ids (#7476)
• Add a configurable DAGs volume mount path for Kubernetes (#8147)
• Add schedulername option for KubernetesPodOperator (#6088)
• Support running git sync container as root (#6312)
• Add extra options for Slack Webhook operator and Slack hook (#9409)
• Monkey patch greenlet celery pools (#8559)
• Decrypt secrets from SystemsManagerParameterStoreBackend (#9214)
• Prevent clickable sorting on non sortable columns in TI view (#8681)
• Make hive macros py3 compatible (#8598)
• Fix SVG tooltip positioning with custom scripting (#8269)
• Avoid unnecessary sleep to maintain local task job heart rate (#6553)
• Include some missing RBAC roles on User and Viewer roles (#9133)
• Show Dag's Markdown docs on Tree View (#9448)
• Improved compatibility with Python 3.5+ - Convert signal.SIGTERM to int (#9207)
• Add 'main' param to template_fields in DataprocSubmitPySparkJobOperator (#9154)
• Make it possible to silence warnings from Airflow (#9208)
• Remove redundant count query in BaseOperator.clear() (#9362)
• Fix DB migration message (#8988)
• Fix awkward log info in dbapi_hook (#8482)
• Fix Celery default to no longer allow pickle (#7205)
• Further validation that only task commands are run by executors (#9240)
• Remove vendored nvd3 and slugify libraries (#9136)
• Enable configurable git sync depth (#9094)
• Reduce the required resources for the Kubernetes's sidecar (#6062)
• Refactor K8S codebase with k8s API models (#5481)
• Move k8sexecutor out of contrib to closer match master (#8904)
• Allow filtering using "event" and "owner" in "Log" view (#4881)
• Add Yandex.Cloud custom connection to 1.10 (#8791)
• Add table-hover css class to DAGs table (#5033)
• Show un/pause errors in dags view. (#7669)
• Restructure database queries on /home (#4872)
• Add Cross Site Scripting defence (#6913)
• Make Gantt tooltip the same as Tree and Graph view (#8220)
• Add config to only delete worker pod on task failure (#7507)(#8312)
• Remove duplicate error message on chart connection failure (#8476)
• Remove default value spark_binary (#8508)
• Expose Airflow Webserver Port in Production Docker Image (#8228)
• Commit after each alembic migration (#4797)
• KubernetesPodOperator fixes and test (#6524)
• Docker Image: Add ADDITIONAL_AIRFLOW_EXTRAS (#9032)
• Docker Image: Add ADDITIONAL_PYTHON_DEPS (#9031)
• Remove httplib2 from Google requirements (#9194)
• Merging multiple sql operators (#9124)
• Adds hive as extra in pyhive dependency (#9075)
• Change default auth for experimental backend to deny_all (#9611)
• Restrict changing XCom values from the Webserver (#9614)
• Add repr for DagTag so tags display properly in /dagmodel/show (#8719)
• Functionality to shuffle HMS connections used by HiveMetastoreHook facilitating load balancing (#9280)
• Expose SQLAlchemy's connect_args and make it configurable (#6478)

Doc only changes

• Add docs on using DAGRun.conf (#9578)
• Enforce code-block directives in doc (#9443)
• Carefully parse warning messages when building documentation (#8693)
• Make KubernetesPodOperator clear in docs (#8444)
• Improve language in Pod Mutation Hook docs (#8445)
• Fix formatting of Pool docs in concepts.rst (#8443)
• Make doc clearer about Airflow Variables using Environment Variables (#8427)
• Fix pools doc for LocalExecutor (#7643)
• Add Local and Sequential Executors to Doc (#8084)
• Add documentation for CLI command airflow dags test (#8251)
• Fix typo in DAG Serialization documentation (#8317)
• Add scheduler in production section (#7351)
• Add a structural dag validation example (#6727)
• Adding Task re-run documentation (#6295)
• Fix outdated doc on settings.policy (#7532)
• Add docs about reload_on_plugin_change opiton (#9575)
• Add copy button to Code Blocks in Airflow Docs (#9450)
• Update commands in docs for v1.10+ (#9585)
• Add more info on dry-run CLI option (#9582)
• Document default timeout value for SSHOperator (#8744)
• Fix docs on creating CustomOperator (#8678)
• Enhanced documentation around Cluster Policy (#8661)
• Use sphinx syntax in concepts.rst (#7729)
• Update README to remove Py 3.8 limitation for Master (#9451)
• Add note about using dag_run.conf in BashOperator (#9143)
• Improve tutorial - Include all imports statements (#8670)
• Added more precise Python requirements to README.md (#8455)
• Fix Airflow Stable version in README.md (#9360)
• Update AWS connection example to show how to set from env var (#9191)
• Fix list formatting of plugins doc. (#8873)
• Add 'Version Added' on Secrets Backend docs (#8264)
• Simplify language re roll-your-own secrets backend (#8257)
• Add installation description for repeatable PyPi installation (#8513)
• Add note extra links only render on when using RBAC webserver (#8788)
• Remove unused airflow import from docs (#9274)
• Add PR/issue note in Contribution Workflow Example (#9177)
• Don't use the term "whitelist" - language matters (#9174)
• Add docs to change Colors on the Webserver (#9607)
• Change 'initiate' to 'initialize' in installation.rst (#9619)
• Replace old Variables View Screenshot with new (#9620)
• Replace old SubDag zoom screenshot with new (#9621)
• Update docs about the change to default auth for experimental API (#9617)