github quantumblacklabs/kedro 0.17.5

Release 0.17.5

Major features and improvements

  • Added new CLI group registry, with the associated commands kedro registry list and kedro registry describe, to replace kedro pipeline list and kedro pipeline describe.
  • Added support for dependency management at a modular pipeline level. When a pipeline with requirements.txt is packaged, its dependencies are embedded in the modular pipeline wheel file. Upon pulling the pipeline, Kedro will append dependencies to the project's requirements.in. More information is available in our documentation.
  • Added support for bulk packaging/pulling modular pipelines using kedro pipeline package/pull --all and pyproject.toml.
  • Removed cli.py from the Kedro project template. By default all CLI commands, including kedro run, are now defined on the Kedro framework side. These can be overridden in turn by a plugin or a cli.py file in your project. A packaged Kedro project will respect the same hierarchy when executed with python -m my_package.
  • Removed .ipython/profile_default/startup/ from the Kedro project template in favour of .ipython/profile_default/ipython_config.py and the kedro.extras.extensions.ipython.
  • Added support for dill backend to PickleDataSet.
  • Imports are now refactored at kedro pipeline package and kedro pipeline pull time, so that aliasing a modular pipeline doesn't break it.
  • Added the following new datasets to support basic Experiment Tracking:
Type Description Location
tracking.MetricsDataSet Dataset to track numeric metrics for experiment tracking kedro.extras.datasets.tracking
tracking.JSONDataSet Dataset to track data for experiment tracking kedro.extras.datasets.tracking

Bug fixes and other changes

  • Bumped minimum required fsspec version to 2021.04.
  • Fixed the kedro install and kedro build-reqs flows when uninstalled dependencies are present in a project's settings.py, context.py or hooks.py (Issue #829).
  • Imports are now refactored at kedro pipeline package and kedro pipeline pull time, so that aliasing a modular pipeline doesn't break it.
  • Pinned dynaconf to <3.1.6 because the method signature for _validate_items changed which is used in Kedro.

Minor breaking changes to the API

Upcoming deprecations for Kedro 0.18.0

  • kedro pipeline list and kedro pipeline describe are being deprecated in favour of new commands kedro registry list and kedro registry describe.
  • kedro install is being deprecated in favour of using pip install -r src/requirements.txt to install project dependencies.

Thanks for supporting contributions

Moussa Taifi,
Deepyaman Datta

9 days ago