github tensorflow/tfx 0.15.0
Release 0.15.0

latest releases: v1.14.0, v1.14.0-rc0, v1.13.0...
4 years ago

Version 0.15.0

Major Features and Improvements

  • Offered unified CLI for tfx pipeline actions on various orchestrators
    including Apache Airflow, Apache Beam and Kubeflow.
  • Polished experimental interactive notebook execution and visualizations so
    they are ready for use.
  • Added BulkInferrer component to TFX pipeline, and corresponding offline
    inference taxi pipeline.
  • Introduced ImporterNode as a special TFX node to register external resource
    into MLMD so that downstream nodes can use as input artifacts. An example
    taxi_pipeline_importer.py enabled by ImporterNode was added to showcase
    the user journey of user-provided schema (issue #571).
  • Added experimental support for TFMA fairness indicator thresholds.
  • Demonstrated DirectRunner multi-core processing in Chicago Taxi example,
    including Airflow and Beam.
  • Introduced PipelineConfig and BaseComponentConfig to control the
    platform specific settings for pipelines and components.
  • Added a custom Executor of Pusher to push model to BigQuery ML for serving.
  • Added KubernetesComponentLauncher to support launch ExecutorContainerSpec in a
    Kubernetes cluster.
  • Made model validator executor forward compatible with TFMA change.
  • Added Iris flowers classification example.
  • Added support for serialization and deserialization of components.
  • Made component launcher extensible to support launching components on
    multiple platforms.
  • Simplified component package names.
  • Introduced BaseNode as the base class of any node in a TFX pipeline DAG.
  • Added docker component launcher to launch container component.
  • Added support for specifying pipeline root in runtime when run on
    KubeflowDagRunner. A default value can be provided when constructing the TFX
    pipeline.
  • Added basic span support in ExampleGen to ingest file based data sources
    that can be updated regularly by upstream.
  • Branched serving examples under chicago_taxi_pipeline/ from chicago_taxi/
    example.
  • Supported beam arg 'direct_num_workers' for multi-processing on local.
  • Improved naming of standard component inputs and outputs.
  • Improved visualization functionality in the experimental TFX notebook
    interface.
  • Allowed users to specify output file format when compiling TFX pipelines
    using KubeflowDagRunner.
  • Introduced ResolverNode as a special TFX node to resolve input artifacts for
    downstream nodes. ResolverNode is a convenient way to wrap TFX Resolver, a
    logical unit for resolving input artifacts.
  • Added cifar-10 example to demonstrate image classification.
  • Added container builder feature in the CLI tool for container-based custom
    python components. This is specifically for the Kubeflow orchestration
    engine, which requires containers built with the custom python code.
  • Demonstrated DirectRunner multi-core processing in Chicago Taxi example,
    including Airflow and Beam.
  • Added Kubeflow artifact visualization of inputs, outputs and
    execution properties for components using a Markdown file. Added Tensorboard
    to Trainer components as well.

Bug fixes and other changes

  • Bumped test dependency to kfp (Kubeflow Pipelines SDK) to be at version
    0.1.31.2.
  • Fixed trainer executor to correctly make transform_output optional.
  • Updated Chicago Taxi example dependency tensorflow to version >=1.14.0.
  • Updated Chicago Taxi example dependencies tensorflow-data-validation,
    tensorflow-metadata, tensorflow-model-analysis, tensorflow-serving-api, and
    tensorflow-transform to version >=0.14.
  • Updated Chicago Taxi example dependencies to Beam 2.14.0, Flink 1.8.1, Spark
    2.4.3.
  • Adopted new recommended way to access component inputs/outputs as
    component.outputs['output_name'] (previously, the syntax was
    component.outputs.output_name).
  • Updated Iris example to skip transform and use Keras model.
  • Fixed the check for input artifact existence in base driver.
  • Fixed bug in AI Platform Pusher that prevents pushes after first model, and
    not being marked as default.
  • Replaced all usage of deprecated tensorflow.logging with absl.logging.
  • Used special user agent for all HTTP requests through googleapiclient and
    apitools.
  • Transform component updated to use tf.compat.v1 according to the TF 2.0
    upgrading procedure.
  • TFX updated to use tf.compat.v1 according to the TF 2.0 upgrading
    procedure.
  • Added Kubeflow local example pipeline that executes components in-cluster.
  • Fixed a bug that prevents updating execution type.
  • Fixed a bug in model validator driver that reads across pipeline boundaries
    when resolving latest blessed model.
  • Depended on apache-beam[gcp]>=2.16,<3
  • Depended on ml-metadata>=0.15,<0.16
  • Depended on tensorflow>=1.15,<3
  • Depended on tensorflow-data-validation>=0.15,<0.16
  • Depended on tensorflow-model-analysis>=0.15.2,<0.16
  • Depended on tensorflow-transform>=0.15,<0.16
  • Depended on 'tfx_bsl>=0.15.1,<0.16'
  • Made launcher return execution information, containing populated inputs,
    outputs, and execution id.
  • Updated the default configuration for accessing MLMD from pipelines running
    in Kubeflow.
  • Updated Airflow developer tutorial
  • CSVExampleGen: started using the CSV decoding utilities in tfx-bsl
    (tfx-bsl>=0.15.2)
  • Added documentation for Fairness Indicators.

Deprecations

  • Deprecated component_type in favor of type.
  • Deprecated component_id in favor of id.
  • Move beam_pipeline_args out of additional_pipeline_args as top level
    pipeline param
  • Deprecated chicago_taxi folder, beam setup scripts and serving examples are
    moved to chicago_taxi_pipeline folder.

Breaking changes

  • Moved beam setup scripts from examples/chicago_taxi/ to
    examples/chicago_taxi_pipeline/
  • Moved interactive notebook classes into tfx.orchestration.experimental
    namespace.
  • Starting from 1.15, package tensorflow comes with GPU support. Users
    won't need to choose between tensorflow and tensorflow-gpu. If any GPU
    devices are available, processes spawned by all TFX components will try to
    utilize them; note that in rare cases, this may exhaust the memory of the
    device(s).
  • Caveat: tensorflow 2.0.0 is an exception and does not have GPU
    support. If tensorflow-gpu 2.0.0 is installed before installing
    tfx, it will be replaced with tensorflow 2.0.0.
    Re-install tensorflow-gpu 2.0.0 if needed.
  • Caveat: MLMD schema auto-upgrade is now disabled by default. For users who
    upgrades from 0.13 and do not want to lose the data in MLMD, please refer to
    MLMD documentation
    for guide to upgrade or downgrade MLMD database. Users who upgraded from TFX
    0.14 should not be affected since there is not schema change between these
    two versions.

For pipeline authors

  • Deprecated the usage of tf.contrib.training.HParams in Trainer as it is
    deprecated in TF 2.0. User module relying on member method of that class
    will not be supported. Dot style property access will be the only supported
    style from now on.
  • Any SavedModel produced by tf.Transform <=0.14 using any tf.contrib ops
    (or tf.Transform ops that used tf.contrib ops such as tft.quantiles,
    tft.bucketize, etc.) cannot be loaded with TF 2.0 since the contrib library
    has been removed in 2.0. Please refer to this [issue]
    (#838).

For component authors

Documentation updates

  • Added conceptual info on Artifacts to guide/index.md

Don't miss a new tfx release

NewReleases is sending notifications on new releases.