Version 0.15.0rc0
Major Features and Improvements
- Offered unified CLI for tfx pipeline actions on various orchestrators
including Apache Airflow, Apache Beam and Kubeflow. - Polished experimental interactive notebook execution and visualizations
so they are ready for use. - Added BulkInferrer component to TFX pipeline, and corresponding offline
inference taxi pipeline. - Introduced ImporterNode as a special TFX node to register external resource
into MLMD so that downstream nodes can use as input artifacts. An example
taxi_pipeline_importer.py
enabled by ImporterNode was added to showcase
the user journey of user-provided schema (issue #571). - Added experimental support for TFMA fairness indicator thresholds.
- Demonstrated DirectRunner multi-core processing in Chicago Taxi example,
including Airflow and Beam. - Made model validator executor forward compatible with TFMA change.
- Added Iris flowers classification example.
- Added support for serialization and deserialization of components.
- Made component launcher extensible to support launching components on
multiple platforms. - Added option to use fixed Schema artifact for ExampleValidator, Transform
and Trainer. - Simplified component package names.
- Introduced BaseNode as the base class of any node in a TFX pipeline DAG.
- Added docker component launcher to launch container component.
- Added support for specifying pipeline root in runtime when run on KubeflowDagRunner.
A default value can be provided when constructing the TFX pipeline. - Added basic span support in ExampleGen to ingest file based data sources
that can be updated regularly by upstream. - Branched serving examples under chicago_taxi_pipeline/ from
chicago_taxi/ example. - Supported beam arg 'direct_num_workers' for multi-processing on local.
- Improved naming of standard component inputs and outputs.
- Improved visualization functionality in the experimental TFX notebook
interface. - Allowed users to specify output file format when compiling TFX pipelines
using KubeflowDagRunner. - Introduced ResolverNode as a special TFX node to resolve input artifacts for
downstream nodes. ResolverNode is a convenient way to wrap TFX Resolver, a
logical unit for resolving input artifacts. - Added cifar-10 example to demonstrate image classification.
- Added container builder feature in the CLI tool for container-based custom
python components. This is specifically for the Kubeflow orchestration
engine, which requires containers built with the custom python code. - Demonstrated DirectRunner multi-core processing in Chicago Taxi example,
including Airflow and Beam.
Bug fixes and other changes
- Bumped test dependency to kfp (Kubeflow Pipelines SDK) to
be at version 0.1.31.2. - Fixed trainer executor to correctly make
transform_output
optional. - Updated Chicago Taxi example dependency tensorflow to version >=1.14.0.
- Updated Chicago Taxi example dependencies tensorflow-data-validation,
tensorflow-metadata, tensorflow-model-analysis, tensorflow-serving-api, and
tensorflow-transform to version >=0.14. - Updated Chicago Taxi example dependencies to Beam 2.14.0, Flink 1.8.1, Spark
2.4.3. - Adopted new recommended way to access component inputs/outputs as
component.outputs['output_name']
(previously, the syntax was
component.outputs.output_name
). - Updated Iris example to skip transform and use Keras model.
- Fixed the check for input artifact existence in base driver.
- Fixed bug in AI Platform Pusher that prevents pushes after first model,
and not being marked as default. - Replaced all usage of deprecated
tensorflow.logging
withabsl.logging
. - Used special user agent for all HTTP requests through
googleapiclient and apitools. - Transform component updated to use
tf.compat.v1
according to the TF 2.0
upgrading procedure. - TFX updated to use
tf.compat.v1
according to the TF 2.0 upgrading
procedure. - Added Kubeflow simple example that executes all components in-cluster.
- Fixed a bug that prevents updating execution type.
- Depended on
apache-beam[gcp]>=2.16,<3
- Depended on
ml-metadata>=0.15,<0.16
- Depended on
tensorflow>=1.15,<3
- Depended on
tensorflow-data-validation>=0.15,<0.16
- Depended on
tensorflow-model-analysis>=0.15.2,<0.16
- Depended on
tensorflow-transform>=0.15,<0.16
- Depended on 'tfx_bsl>=0.15.1,<0.16'
Deprecations
- Deprecated component_type in favor of type.
- Deprecated component_id in favor of id.
- Move beam_pipeline_args out of additional_pipeline_args as top level
pipeline param - Deprecated chicago_taxi folder, beam setup scripts and serving examples are
moved to chicago_taxi_pipeline folder.
Breaking changes
- Moved beam setup scripts from examples/chicago_taxi/ to
examples/chicago_taxi_pipeline/ - Moved interactive notebook classes into
tfx.orchestration.experimental
namespace.
For pipeline authors
- Deprecated the usage of
tf.contrib.training.HParams
in Trainer as it is
deprecated in TF 2.0. User module relying on member method of that class
will not be supported. Dot style property access will be the only supported
style from now on.
For component authors
Documentation updates
- Added conceptual info on Artifacts to guide/index.md