kedro-org/kedro 0.16.2

on GitHub

Major features and improvements

• Added the following new datasets.

• kedro install will now compile project dependencies (by running kedro build-reqs behind the scenes) before the installation if the src/requirements.in file doesn't exist.
• Added only_nodes_with_namespace in Pipeline class to filter only nodes with a specified namespace.
• Added the kedro pipeline delete command to help delete unwanted or unused pipelines (it won't remove references to the pipeline in your create_pipelines() code).
• Added the kedro pipeline package command to help package up a modular pipeline. It will bundle up the pipeline source code, tests, and parameters configuration into a .whl file.

Bug fixes and other changes

• Improvement in DataCatalog:
  • Introduced regex filtering to the DataCatalog.list() method.
  • Non-alphanumeric characters (except underscore) in dataset name are replaced with __ in DataCatalog.datasets, for ease of access to transcoded datasets.
• Improvement in Datasets:
  • Improved initialization speed of spark.SparkHiveDataSet.
  • Improved S3 cache in spark.SparkDataSet.
  • Added support of options for building pyarrow table in pandas.ParquetDataSet.
• Improvement in kedro build-reqs CLI command:
  • kedro build-reqs is now called with -q option and will no longer print out compiled requirements to the console for security reasons.
  • All unrecognized CLI options in kedro build-reqs command are now passed to pip-compile call (e.g. kedro build-reqs --generate-hashes).
• Improvement in kedro jupyter CLI command:
  • Improved error message when running kedro jupyter notebook, kedro jupyter lab or kedro ipython with Jupyter/IPython dependencies not being installed.
  • Fixed %run_viz line magic for showing kedro viz inside a Jupyter notebook. For the fix to be applied on existing Kedro project, please see the migration guide.
  • Fixed the bug in IPython startup script (issue 298).
• Documentation improvements:
  • Updated community-generated content in FAQ.
  • Added find-kedro and kedro-static-viz to the list of community plugins.
  • Add missing pillow.ImageDataSet entry to the documentation.

Breaking changes to the API

Migration guide from Kedro 0.16.1 to 0.16.2

Guide to apply the fix for %run_viz line magic in existing project

Even though this release ships a fix for project generated with kedro==0.16.2, after upgrading, you will still need to make a change in your existing project if it was generated with kedro>=0.16.0,<=0.16.1 for the fix to take effect. Specifically, please change the content of your project's IPython init script located at .ipython/profile_default/startup/00-kedro-init.py with the content of this file. You will also need kedro-viz>=3.3.1.

Thanks for supporting contributions

Miguel Rodriguez Gutierrez, Joel Schwarzmann, w0rdsm1th, Deepyaman Datta, Tam-Sanh Nguyen, Marcus Gawronsky