Release 0.18.13

Major features and improvements

• Added support for Python 3.11. This includes tackling challenges like dependency pinning and test adjustments to ensure a smooth experience. Detailed migration tips are provided below for further context.
• Added new OmegaConfigLoader features:
  • Allowed registering of custom resolvers to OmegaConfigLoader through CONFIG_LOADER_ARGS.
  • Added support for global variables to OmegaConfigLoader.
• Added kedro catalog resolve CLI command that resolves dataset factories in the catalog with any explicit entries in the project pipeline.
• Implemented a flat conf/ structure for modular pipelines, and accordingly, updated the kedro pipeline create and kedro catalog create command.
• Updated new Kedro project template and Kedro starters:
  • Change Kedro starters and new Kedro projects to use OmegaConfigLoader.
  • Converted setup.py in new Kedro project template and Kedro starters to pyproject.toml and moved flake8 configuration
    to dedicated file .flake8.
  • Updated the spaceflights starter to use the new flat conf/ structure.

Bug fixes and other changes

• Updated OmegaConfigLoader to ignore config from hidden directories like .ipynb_checkpoints.

Documentation changes

• Revised the data section to restructure beginner and advanced pages about the Data Catalog and datasets.
• Moved contributor documentation to the GitHub wiki.
• Updated example of using generator functions in nodes.
• Added migration guide from the ConfigLoader and the TemplatedConfigLoader to the OmegaConfigLoader. The ConfigLoader and the TemplatedConfigLoader are deprecated and will be removed in the 0.19.0 release.

Migration Tips for Python 3.11:

• PyTables on Windows: Users on Windows with Python >=3.8 should note we've pinned pytables to 3.8.0 due to compatibility issues.
• Spark Dependency: We've set an upper version limit for pyspark at <3.4 due to breaking changes in 3.4.
• Testing with Python 3.10: The latest moto version now supports parallel test execution for Python 3.10, resolving previous issues.

Breaking changes to the API

Upcoming deprecations for Kedro 0.19.0

• Renamed abstract dataset classes, in accordance with the Kedro lexicon. Dataset classes ending with "DataSet" are deprecated and will be removed in 0.19.0. Note that all of the below classes are also importable from kedro.io; only the module where they are defined is listed as the location.

• Using the layer attribute at the top level is deprecated; it will be removed in Kedro version 0.19.0. Please move layer inside the metadata -> kedro-viz attributes.

Community contributions

Thanks to Laíza Milena Scheid Parizotto and Jonathan Cohen.