github quantumblacklabs/kedro 0.17.1

Release 0.17.1

Major features and improvements

  • Added env and extra_params to reload_kedro() line magic.
  • Extended the pipeline() API to allow strings and sets of strings as inputs and outputs, to specify when a dataset name remains the same (not namespaced).
  • Added the ability to add custom prompts with regexp validator for starters by repurposing default_config.yml as prompts.yml.
  • Added the env and extra_params arguments to register_config_loader hook.
  • Refactored the way settings are loaded. You will now be able to run:
from kedro.framework.project import settings

print(settings.CONF_ROOT)

Bug fixes and other changes

  • The version of a packaged modular pipeline now defaults to the version of the project package.
  • Added fix to prevent new lines being added to pandas CSV datasets.
  • Fixed issue with loading a versioned SparkDataSet in the interactive workflow.
  • Kedro CLI now checks pyproject.toml for a tool.kedro section before treating the project as a Kedro project.
  • Added fix to DataCatalog::shallow_copy now it should copy layers.
  • kedro pipeline pull now uses pip download for protocols that are not supported by fsspec.
  • Cleaned up documentation to fix broken links and rewrite permanently redirected ones.
  • Added a jsonschema schema definition for the Kedro 0.17 catalog.
  • kedro install now waits on Windows until all the requirements are installed.
  • Exposed --to-outputs option in the CLI, throughout the codebase, and as part of hooks specifications.
  • Fixed a bug where ParquetDataSet wasn't creating parent directories on the fly.
  • Updated documentation.

Breaking changes to the API

  • This release has broken the kedro ipython and kedro jupyter workflows. To fix this, follow the instructions in the migration guide below.

Note: If you're using the ipython extension instead, you will not encounter this problem.

Migration guide

You will have to update the file <your_project>/.ipython/profile_default/startup/00-kedro-init.py in order to make kedro ipython and/or kedro jupyter work. Add the following line before the KedroSession is created:

configure_project(metadata.package_name)  # to add

session = KedroSession.create(metadata.package_name, path)

Make sure that the associated import is provided in the same place as others in the file:

from kedro.framework.project import configure_project  # to add
from kedro.framework.session import KedroSession

Thanks for supporting contributions

Mariana Silva,
Kiyohito Kunii,
noklam,
Ivan Doroshenko,
Zain Patel,
Deepyaman Datta,
Sam Hiscox,
Pascal Brokmeier

latest releases: 0.17.5, 0.17.4, 0.17.3...
6 months ago