github quantumblacklabs/kedro 0.16.2

Major features and improvements

  • Added the following new datasets.
Type Description Location
pandas.AppendableExcelDataSet Works with Excel file opened in append mode kedro.extras.datasets.pandas
tensorflow.TensorFlowModelDataset Works with TensorFlow models using TensorFlow 2.X kedro.extras.datasets.tensorflow
holoviews.HoloviewsWriter Works with Holoviews objects (saves as image file) kedro.extras.datasets.holoviews
  • kedro install will now compile project dependencies (by running kedro build-reqs behind the scenes) before the installation if the src/ file doesn't exist.
  • Added only_nodes_with_namespace in Pipeline class to filter only nodes with a specified namespace.
  • Added the kedro pipeline delete command to help delete unwanted or unused pipelines (it won't remove references to the pipeline in your create_pipelines() code).
  • Added the kedro pipeline package command to help package up a modular pipeline. It will bundle up the pipeline source code, tests, and parameters configuration into a .whl file.

Bug fixes and other changes

  • Improvement in DataCatalog:
    • Introduced regex filtering to the DataCatalog.list() method.
    • Non-alphanumeric characters (except underscore) in dataset name are replaced with __ in DataCatalog.datasets, for ease of access to transcoded datasets.
  • Improvement in Datasets:
    • Improved initialization speed of spark.SparkHiveDataSet.
    • Improved S3 cache in spark.SparkDataSet.
    • Added support of options for building pyarrow table in pandas.ParquetDataSet.
  • Improvement in kedro build-reqs CLI command:
    • kedro build-reqs is now called with -q option and will no longer print out compiled requirements to the console for security reasons.
    • All unrecognized CLI options in kedro build-reqs command are now passed to pip-compile call (e.g. kedro build-reqs --generate-hashes).
  • Improvement in kedro jupyter CLI command:
    • Improved error message when running kedro jupyter notebook, kedro jupyter lab or kedro ipython with Jupyter/IPython dependencies not being installed.
    • Fixed %run_viz line magic for showing kedro viz inside a Jupyter notebook. For the fix to be applied on existing Kedro project, please see the migration guide.
    • Fixed the bug in IPython startup script (issue 298).
  • Documentation improvements:
    • Updated community-generated content in FAQ.
    • Added find-kedro and kedro-static-viz to the list of community plugins.
    • Add missing pillow.ImageDataSet entry to the documentation.

Breaking changes to the API

Migration guide from Kedro 0.16.1 to 0.16.2

Guide to apply the fix for %run_viz line magic in existing project

Even though this release ships a fix for project generated with kedro==0.16.2, after upgrading, you will still need to make a change in your existing project if it was generated with kedro>=0.16.0,<=0.16.1 for the fix to take effect. Specifically, please change the content of your project's IPython init script located at .ipython/profile_default/startup/ with the content of this file. You will also need kedro-viz>=3.3.1.

Thanks for supporting contributions

Miguel Rodriguez Gutierrez, Joel Schwarzmann, w0rdsm1th, Deepyaman Datta, Tam-Sanh Nguyen, Marcus Gawronsky

latest releases: 0.17.5, 0.17.4, 0.17.3...
15 months ago