This is the release note of v2.2.0.

In this release, we drop support for Python 3.5. If you are using Python 3.5, please consider upgrading your Python environment to Python 3.6 or newer, or install older versions of Optuna.

Highlights

Multivariate TPE sampler

TPESampler is updated with an experimental option to enable multivariate sampling. This algorithm captures dependencies among hyperparameters better than the previous algorithm. See #1767 for more details.

Improved AllenNLP support

AllenNLPExecutor supports pruning. It is introduced in the official hyperparameter search guide by AllenNLP. Both AllenNLPExecutor and the guide were written by @himkt. See #1772.

New Features

• Create optuna.visualization.matplotlib (#1756, thanks @ytknzw!)
• Add multivariate TPE sampler (#1767, thanks @kstoneriv3!)
• Support AllenNLPPruningCallback for AllenNLPExecutor (#1772)

Enhancements

• KerasPruningCallback to warn when an evaluation metric does not exist (#1759, thanks @bigbird555!)
• Implement plot_edf and _get_edf_plot with Matplotlib backend (#1760, thanks @ytknzw!)
• Fix exception chaining all over the codebase (#1781, thanks @akihironitta!)
• Add metric alias of rmse for LightGBMTuner (#1807, thanks @upura!)
• Update PyTorch-Lighting minor version (#1813, thanks @nzw0301!)
• Improve TensorBoardCallback (#1814, thanks @sfujiwara!)
• Add metric alias for LightGBMTuner (#1822, thanks @nyanhi!)
• Introduce a new argument to plot all evaluation points by optuna.multi_objective.visualization.plot_pareto_front (#1824, thanks @nzw0301!)
• Add reseed_rng to RandomMultiobjectiveSampler (#1831, thanks @y0z!)

Bug Fixes

• Fix fANOVA for IntLogUniformDistribution (#1788)
• Fix mypy in an environment where some dependencies are installed (#1804)
• Fix WFG._compute() (#1812, thanks @y0z!)
• Fix contour plot error for categorical distributions (#1819, thanks @zchenry!)
• Store CMAES optimizer after splitting into substrings (#1833)
• Add maximize support on CmaEsSampler (#1849)
• Add matplotlib directory to optuna.visualization.__init__.py (#1867)

Installation

• Update setup.py to drop Python 3.5 support (#1818, thanks @harupy!)
• Add Matplotlib to setup.py (#1829, thanks @ytknzw!)

Documentation

• Fix plot_pareto_front preview path (#1808)
• Fix indents of the example of multi_objective.visualization.plot_pareto_front (#1815, thanks @nzw0301!)
• Hide __init__ from docs (#1820, thanks @upura!)
• Explicitly omit Python 3.5 from README.md (#1825)
• Follow-up #1832: alphabetical naming and fixes (#1841)
• Mention isort in the contribution guidelines (#1842)
• Add news sections about introduction of isort (#1843)
• Add visualization.matpltlib to docs (#1847)
• Add sphinx doc comments regarding exceptions in the optimize method (#1857, thanks @yuk1ty!)
• Avoid global study in Study.stop testcode (#1861)
• Fix documents of visualization.is_available (#1869)
• Improve ThresholdPruner example (#1876, thanks @fsmosca!)
• Add logging levels to optuna.logging.set_verbosity (#1884, thanks @nzw0301!)

Examples

• Add XGBoost cross-validation example (#1836, thanks @sskarkhanis!)
• Minor code fix of XGBoost examples (#1844)

Code Fixes

• Add default implementation of get_n_trials (#1568)
• Introduce isort to automatically sort import statements (#1695, thanks @harupy!)
• Avoid using experimental decorator on CmaEsSampler (#1777)
• Remove logger member attributes from PyCmaSampler and CmaEsSampler (#1784)
• Apply blackdoc (#1817)
• Remove TODO (#1821, thanks @sfujiwara!)
• Fix Redis example code (#1826)
• Apply isort to visualization/matplotlib/ and multi_objective/visualization (#1830)
• Move away from .scoring imports (#1864, thanks @norihitoishida!)
• Add experimental decorator to matplotlib.* (#1868)

Continuous Integration

• Disable --cache-from if trigger of docker image build is release (#1791)
• Remove Python 3.5 from CI checks (#1810, thanks @harupy!)
• Update python version in docs (#1816, thanks @harupy!)
• Migrate checks to GitHub Actions (#1838)
• Add option --diff to black (#1840)

Thanks to All the Contributors!

This release was made possible by authors, and everyone who participated in reviews and discussions.

@HideakiImamura, @akihironitta, @bigbird555, @c-bata, @crcrpar, @fsmosca, @g-votte, @harupy, @himkt, @hvy, @keisuke-umezawa, @kstoneriv3, @norihitoishida, @nyanhi, @nzw0301, @sfujiwara, @sile, @sskarkhanis, @toshihikoyanase, @upura, @y0z, @ytknzw, @yuk1ty, @zchenry