apache-airflow 1.10.10

Airflow 1.10.10, 2020-04-09

on Python PyPI

New Features

• [AIRFLOW-7048] Allow user to chose timezone to use in UI (#8046)
• Add Production Docker image support (#7832)
• Get Airflow Variables from Environment Variables (#7923)
• Get Airflow Variables from Hashicorp Vault (#7944)
• Get Airflow Variables from AWS Systems Manager Parameter Store (#7945)
• Get Airflow Variables from GCP Secrets Manager (#7946)
• [AIRFLOW-5705] Add secrets backend and support for AWS SSM / Get Airflow Connections from AWS Parameter Store(#6376)
• [AIRFLOW-7104] Add Secret backend for GCP Secrets Manager / Get Airflow Connections from GCP Secrets Manager (#7795)
• [AIRFLOW-7076] Add support for HashiCorp Vault as Secrets Backend / Get Airflow Connections from Hashicorp Vault (#7741)
• [AIRFLOW-6685] Add ThresholdCheckOperator (#7353)
• [AIRFLOW-7080] Add API endpoint to return a DAG's paused state (#7737)

Improvements

• Use same tooltip for Graph and Tree views for TaskInstances (#8043)
• Allow DateTimePicker in Webserver to actually pick times too (#8034)
• [AIRFLOW-5590] Add run_id to trigger DAG run API response (#6256)
• [AIRFLOW-6695] Can now pass dagrun conf when triggering dags via UI (#7312)
• [AIRFLOW-5336] Add ability to make updating FAB perms on webserver in it optional (#5940)
• [AIRFLOW-1467] Allow tasks to use more than one pool slot (#7160)
• [AIRFLOW-6987] Avoid creating default connections (#7629)
• [AIRFLOW-4175] S3Hook load_file should support ACL policy parameter (#7733)
• [AIRFLOW-4438] Add Gzip compression to S3_hook (#7680)
• Allow setting Airflow Variable values to empty string (#8021)
• Dont schedule dummy tasks (#7880)
• Prevent sequential scan of task instance table (#8014)
• [AIRFLOW-7017] Respect default dag view in trigger dag origin (#7667)
• [AIRFLOW-6837] Limit description length of a Dag on HomePage (#7457)
• [AIRFLOW-6989] Display Rendered template_fields without accessing Dag files (#7633)
• [AIRFLOW-5944] Rendering templated_fields without accessing DAG files (#6788)
• [AIRFLOW-5946] DAG Serialization: Store source code in db (#7217)
• [AIRFLOW-7079] Remove redundant code for storing template_fields (#7750)
• [AIRFLOW-7024] Add the verbose parameter support to SparkSqlOperator (#7676)
• [AIRFLOW-6733] Extend not replace template (#7366)
• [AIRFLOW-7001] Further fix for the MySQL 5.7 UtcDateTime (#7655)
• [AIRFLOW-6014] Handle pods which are preempted & deleted by kubernetes but not restarted (#6606)
• [AIRFLOW-6950] Remove refresh_executor_config from ti.refresh_from_db (#7577)
• [AIRFLOW-7016] Sort dag tags in the UI (#7661)
• [AIRFLOW-6762] Fix link to "Suggest changes on this page" (#7387)
• [AIRFLOW-6948] Remove ASCII Airflow from version command (#7572)
• [AIRFLOW-6767] Correct name for default Athena workgroup (#7394)
• [AIRFLOW-6905] Update pin.svg with new pinwheel (#7524)
• [AIRFLOW-6801] Make use of ImportError.timestamp (#7425)
• [AIRFLOW-6830] Add Subject/MessageAttributes to SNS hook and operator (#7451)
• [AIRFLOW-6630] Resolve handlebars advisory (#7284)
• [AIRFLOW-6945] MySQL 5.7 is used in v1-10-test as an option
• [AIRFLOW-6871] Optimize tree view for large DAGs (#7492)
• [AIRFLOW-7063] Fix dag.clear() slowness caused by count (#7723)
• [AIRFLOW-7023] Remove duplicated package definitions in setup.py (#7675)
• [AIRFLOW-7001] Time zone removed from MySQL TIMSTAMP field inserts
• [AIRFLOW-7105] Unify Secrets Backend method interfaces (#7830)
• Make BaseSecretsBackend.build_path generic (#7948)
• Allow hvac pakage installation using 'hashicorp' extra (#7915)
• Standardize SecretBackend class names (#7846)
• [AIRFLOW-5705] Make AwsSsmSecretsBackend consistent with VaultBackend (#7753)
• [AIRFLOW-7045] Update SQL query to delete RenderedTaskInstanceFields (#8051)
• Handle DST better in Task Instance tool tips (#8104)

Bug Fixes

• BugFix: Show task_id in the Graph View tooltip (#7859)
• [AIRFLOW-6730] Use total_seconds instead of seconds (#7363)
• [AIRFLOW-6167] Escape column name in create table in hive (#6741)
• [AIRFLOW-6628] DAG auto-complete now suggests from all acessible DAGs (#7251)
• [AIRFLOW-7113] Fix gantt render error (#7913)
• [AIRFLOW-6399] Add _access control to validate deserialized DAGs (#7896)
• [AIRFLOW-6399] Serialization: DAG access_control field should be decorated field in DAG serialization (#7879)
• [AIRFLOW-4453] Make behavior of none_failed consistent with documentation (#7464)
• [AIRFLOW-4363] Fix JSON encoding error (#7628)
• [AIRFLOW-6683] Run REST API tests when DAGs are serialized (#7352)
• [AIRFLOW-6704] Copy common TaskInstance attributes from Task (#7324)
• [AIRFLOW-6734] Use configured base_template instead of hard-coding (#7367)
• [AIRFLOW-7098] Simple salesforce release 1.0.0 breaks the build (#7775)
• [AIRFLOW-6062] Executor would only delete workers in its own namespace (#7123)
• [AIRFLOW-7074] Add Permissions to view SubDAGs (#7752)
• [AIRFLOW-7025] Fix SparkSqlHook.run_query to handle its parameter properly (#7677)
• [AIRFLOW-6855] Escape project_dataset_table in SQL query in gcs to bq operator (#7475)
• [AIRFLOW-6949] Respect explicit conf to SparkSubmitOperator (#7575)
• [AIRFLOW-6588] write_stdout and json_format are boolean (#7199)
• [AIRFLOW-3439] Decode logs with 'utf-8' (#4474)
• [AIRFLOW-6878] Fix misconfigured default value for kube_client_request_args
• [AIRFLOW-5167] Update dependencies for GCP packages (#7116)
• [AIRFLOW-6821] Success callback not called when task marked as success from UI (#7447)
• [AIRFLOW-6740] Remove Undocumented, deprecated, dysfunctional PROXY_FIX_NUM_PROXIES (#7359)
• [AIRFLOW-6728] Change various DAG info methods to POST (#7364)
• [AIRFLOW-6997] Make sure worker pods initcontainers obtain env vars from config (#7663)
• [AIRFLOW-7062] Fix pydruid release breaking the build (#7720)
• [AIRFLOW-6040] ReadTimoutError in KubernetesExecutor should not raise exception (#7616)
• [AIRFLOW-6943] Fix utf-8 encodeed description in DAG in Python 2 (#7567)
• [AIRFLOW-6892] Fix broken non-wheel releases (#7514)
• [AIRFLOW-6789] BugFix: Fix Default Worker concurrency (#7494)
• [AIRFLOW-6840] Bump up version of future (#7471)
• [AIRFLOW-5705] Fix bugs in AWS SSM Secrets Backend (#7745)
• [AIRFLOW-5705] Fix bug in Secrets Backend (#7742)
• Fix CloudSecretsManagerBackend invalid connections_prefix (#7861)
• [AIRFLOW-7045] BugFix: DebugExecutor fails to change task state. (#8073)
• BugFix: Datetimepicker is stuck on the UI (#8092)
• [AIRFLOW-5277] Gantt chart respects per-user the Timezone UI setting (#8096)
• Fix timezones displayed in Task Instance tooltip (#8103)
• BugFix: Fix writing & deleting Dag Code for Serialized DAGs (#8151)
• Make the default TI pool slots '1' (#8153)
• Fix 500 error in Security screens (#8165)
• Fix Viewing Dag Code for Stateless Webserver (#8159)
• Fix issue with SQLAlchemy 1.3.16 (#8230)

Misc/Internal

• Fix Flaky TriggerDAG UI test (#8022)
• Remove unnecessary messages in CI (#7951)
• Fixes too high parallelism in CI (#7947)
• Install version is not persistent in breeze (#7914)
• Fixed automated check for image rebuild (#7912)
• Move Dockerfile to Dockerfile.ci (#7829)
• Generate requirements are now sorted (#8040)
• Change name of the common environment initialization function (#7805)
• Requirements now depend on python version (#7841)
• Bring back reset db explicitly called at CI entry (#7798)
• Fixes unclean installation of Airflow 1.10 (#7796)
• Change name of the common environment initialization function (#7805)
• [AIRFLOW-7029] Use separate docker image for running license check (#7678)
• [AIRFLOW-5842] Swtch to Debian buster image as a base (#7647)
• [AIRFLOW-5828] Move build logic out from hooks/build (#7618)
• [AIRFLOW-6839] Even more mypy speed improvements (#7460)
• [AIRFLOW-6820] split breeze into functions (#7433)
• [AIRFLOW-7097] Install gcloud beta componensts in CI image (#7772)
• [AIRFLOW-7018] fixing travis's job name escaping problem (#7668)
• [AIRFLOW-7054] Breeze has an option now to reset db at entry (#7710)
• [AIRFLOW-7005] Added exec command to Breeze (#7649)
• [AIRFLOW-7015] Detect Dockerhub repo/user when building on Dockerhub (#7673)
• [AIRFLOW-6727] Fix minor bugs in Release Management scripts (#7355)
• [AIRFLOW-7013] Automated check if Breeze image needs to be pulled (#7656)
• [AIRFLOW-7010] Skip in-container checks for Dockerhub builds (#7652)
• [AIRFLOW-7011] Pin JPype release to allow to build 1.10 images
• [AIRFLOW-7006] Fix missing +e in Breeze script (#7648)
• [AIRFLOW-6979] Fix breeze test-target specific test param issue (#7614)
• [AIRFLOW-6932] Add restart-environment command to Breeze
• [AIRFLOW-6919] Make Breeze DAG-test friedly (#7539)
• [AIRFLOW-6838] Introduce real subcommands for Breeze (#7515)
• [AIRFLOW-6763] Make systems tests ready for backport tests (#7389)
• [AIRFLOW-6866] Fix wrong export for Mac on Breeze (#7485)
• [AIRFLOW-6842] Skip fixing ownership on Mac (#7469)
• [AIRFLOW-6841] Fixed unbounded variable on Mac (#7465)
• [AIRFLOW-7067] Pinned version of Apache Airflow (#7730)
• [AIRFLOW-7058] Add support for different DB versions (#7717)
• [AIRFLOW-7002] Get rid of yaml "parser" in bash (#7646)
• [AIRFLOW-6972] Shorter frequently used commands in Breeze (#7608)

Doc only changes

• Fix typo for store_serialized_dags config (#7952)
• Fix broken link in README.md (#7893)
• Separate supported Postgres versions with comma (#7892)
• Fix grammar in setup.py (#7877)
• Add Jiajie Zhong to committers list (#8047)
• Update Security doc for 1.10.* for auth backends (#8072)
• Fix Example in config_templates for Secrets Backend (#8074)
• Add backticks in IMAGES.rst command description (#8075)
• Change version_added for store_dag_code config (#8076)
• [AIRFLOW-XXXX] Remove duplicate docs
• [AIRFLOW-XXXX] Remove the defunct limitation of Dag Serialization (#7716)
• [AIRFLOW-XXXX] Add prerequisite tasks for all GCP operators guide (#6049)
• [AIRFLOW-XXXX] Simplify AWS/Azure/Databricks operators listing (#6047)
• [AIRFLOW-XXXX] Add external reference to all GCP operator guide (#6048)
• [AIRFLOW-XXXX] Simplify GCP operators listing
• [AIRFLOW-XXXX] Simplify Qubole operators listing
• [AIRFLOW-XXXX] Add autogenerated TOC (#6038)
• [AIRFLOW-XXXX] Create "Using the CLI" page (#5823)
• [AIRFLOW-XXXX] Group references in one section (#5776)
• [AIRFLOW-XXXX] Fix analytics doc (#5885)
• [AIRFLOW-XXXX] Add S3 Logging section (#6039)
• [AIRFLOW-XXXX] Move Azure Logging section above operators (#6040)
• [AIRFLOW-XXXX] Update temp link to a fixed link (#7715)
• [AIRFLOW-XXXX] Add Updating.md section for 1.10.9 (#7385)
• [AIRFLOW-XXXX] Remove duplication in BaseOperator docstring (#7321)
• [AIRFLOW-XXXX] Update tests info in CONTRIBUTING.rst (#7466)
• [AIRFLOW-XXXX] Small BREEZE.rst update (#7487)
• [AIRFLOW-XXXX] Add instructions for logging to localstack S3 (#7461)
• [AIRFLOW-XXXX] Remove travis config warnings (#7467)
• [AIRFLOW-XXXX] Add communication chapter to contributing (#7204)
• [AIRFLOW-XXXX] Add known issue - example_dags/init.py (#7444)
• [AIRFLOW-XXXX] Fix breeze build-docs (#7445)
• [AIRFLOW-XXXX] Less verbose docker builds
• [AIRFLOW-XXXX] Speed up mypy runs (#7421)
• [AIRFLOW-XXXX] Fix location of kubernetes tests (#7373)
• [AIRFLOW-XXXX] Remove quotes from domains in Google Oauth (#4226)
• [AIRFLOW-XXXX] Add explicit info about JIRAs for code-related PRs (#7318)
• [AIRFLOW-XXXX] Fix typo commiter => committer (#7392)
• [AIRFLOW-XXXX] Remove duplicated paragraph in docs (#7662)
• Fix reference to KubernetesPodOperator (#8100)