This is the release note of v2.3.0.

Highlights

Multi-objective TPE sampler

TPE sampler now supports multi-objective optimization. This new algorithm is implemented in optuna.multi_objective and used viaoptuna.multi_objective.samplers.MOTPEMultiObjectiveSampler. See #1530 for the details.

LightGBMTunerCV returns the best booster

The best booster of LightGBMTunerCV can now be obtained in the same way as the LightGBMTuner. See #1609 and #1702 for details.

PyTorch Lightning v1.0 support

The integration with PyTorch Lightning v1.0 is available. The pruning feature of Optuna can be used with the new version of PyTorch Lightning using optuna.integration.PyTorchLightningPruningCallback. See #597 and #1926 for details.

RAPIDS + Optuna example

An example to illustrate how to use RAPIDS with Optuna is available. You can use this example to harness the computational power of the GPU along with Optuna.

New Features

• Introduce Multi-objective TPE to optuna.multi_objective.samplers (#1530, thanks @y0z!)
• Return LGBMTunerCV booster (#1702, thanks @nyanhi!)
• Implement plot_intermediate_values and _get_intermediate_plot with Matplotlib backend (#1762, thanks @ytknzw!)
• Implement plot_optimization_history and _get_optimization_history_plot with Matplotlib backend (#1763, thanks @ytknzw!)
• Implement plot_parallel_coordinate and _get_parallel_coordinate_plot with Matplotlib backend (#1764, thanks @ytknzw!)
• Improve MLflow callback functionality: allow nesting, and attached study attrs (#1918, thanks @drobison00!)

Enhancements

• Copy datasets before objective evaluation (#1805)
• Fix 'Mean of empty slice' warning (#1927, thanks @carefree0910!)
• Add reseed_rng to NSGAIIMultiObjectiveSampler (#1938)
• Add RDB support to MoTPEMultiObjectiveSampler (#1978)

Bug Fixes

• Add some jitters in _MultivariateParzenEstimators (#1923, thanks @kstoneriv3!)
• Fix plot_contour (#1929, thanks @carefree0910!)
• Fix return type of the multivariate TPE samplers (#1955, thanks @nzw0301!)
• Fix StudyDirection of mape in LightGBMTuner (#1966)

Documentation

• Add explanation for most module-level reference pages (#1850, thanks @tktran!)
• Revert module directives (#1873)
• Remove with_trace method from docs (#1882, thanks @i-am-jeetu!)
• Add CuPy to projects using Optuna (#1889)
• Add more sphinx doc comments (#1894, thanks @yuk1ty!)
• Fix a broken link in matplotlib.plot_edf (#1899)
• Fix broken links in README.md (#1901)
• Show module paths in optuna.visualization and optuna.multi_objective.visualization (#1902)
• Add a short description to the example in FAQ (#1903)
• Embed plot_edf figure in documentation by using matplotlib plot directive (#1905, thanks @harupy!)
• Fix plotly figure iframe paths (#1906, thanks @harupy!)
• Update docstring of CmaEsSampler (#1909)
• Add matplotlib.plot_intermediate_values figure to doc (#1933, thanks @harupy!)
• Add matplotlib.plot_optimization_history figure to doc (#1934, thanks @harupy!)
• Make code example of MOTPEMultiObjectiveSampler executable (#1953)
• Add Raises comments to samplers (#1965, thanks @yuk1ty!)

Examples

• Make src comments more descriptive in examples/pytorch_lightning_simple.py (#1878, thanks @iamshnoo!)
• Add an external project in Optuna examples (#1888, thanks @resnant!)
• Add RAPIDS + Optuna simple example (#1924, thanks @Nanthini10!)
• Apply follow-up of #1924 (#1960)

Tests

• Fix RDB test to avoid deadlock when creating study (#1919)
• Add a test to verify nest_trials for MLflowCallback works properly (#1932, thanks @harupy!)
• Add a test to verify tag_study_user_attrs for MLflowCallback works properly (#1935, thanks @harupy!)

Code Fixes

• Fix typo (#1900)
• Refactor Study.optimize (#1904)
• Refactor Study.trials_dataframe (#1907)
• Add variable annotation to optuna/logging.py (#1920, thanks @akihironitta!)
• Fix duplicate stack traces (#1921, thanks @akihironitta!)
• Remove _log_normal_cdf (#1922, thanks @kstoneriv3!)
• Convert comment style type hints (#1950, thanks @akihironitta!)
• Align the usage of type hints and instantiation of dictionaries (#1956, thanks @akihironitta!)

Continuous Integration

• Run documentation build and doctest in GitHub Actions (#1891)
• Resolve conflict of job-id of GitHub Actions workflows (#1898)
• Pin mypy==0.782 (#1913)
• Run allennlp_jsonnet.py on GitHub Actions (#1915)
• Fix for PyTorch Lightning 1.0 (#1926)
• Check blackdoc in CI (#1958)
• Fix path for store_artifacts step in document CircleCI job (#1962, thanks @harupy!)

Other

• Fix how to check the format, coding style, and type hints (#1755)
• Fix typo (#1968, thanks @nzw0301!)

Thanks to All the Contributors!

This release was made possible by authors, and everyone who participated in reviews and discussions.

@Crissman, @HideakiImamura, @Nanthini10, @akihironitta, @c-bata, @carefree0910, @crcrpar, @drobison00, @harupy, @himkt, @hvy, @i-am-jeetu, @iamshnoo, @keisuke-umezawa, @kstoneriv3, @nyanhi, @nzw0301, @resnant, @sile, @smly, @tktran, @toshihikoyanase, @y0z, @ytknzw, @yuk1ty