pypi optuna 4.0.0
v4.0.0
on Python PyPI

latest release: 4.1.0
3 months ago

Here is the release note of v4.0.0. Please also check out the release blog post.

If you want to update the Optuna version of your existing projects to v4.0, please see the migration guide.

We have also published blog posts about the development items. Please check them out!

Highlights

Official Release of Feature-Sharing Platform OptunaHub

We officially released OptunaHub, a feature-sharing platform for Optuna. A large number of optimization and visualization algorithms are available in OptunaHub. Contributors can easily register their methods and deliver them to Optuna users around the world.

Please also read the OptunaHub release blog post.

optunahub

Enhanced Experiment Management Feature: Official Support of Artifact Store

Artifact Store is a file management feature for files generated during optimization, dubbed artifacts. In Optuna v4.0, we stabilized the existing file upload API and further enhanced the usability of Artifact Store by adding some APIs such as the artifact download API. We also added features to show JSONL and CSV files on Optuna Dashboard in addition to the existing support for images, audio, and video. With this official support, the API backward compatibility will be guaranteed.

For more details, please check the blog post.

artifact

JournalStorage: Official Support of Distributed Optimization via Network File System

JournalStorage is a new Optuna storage experimentally introduced in Optuna v3.1 (see the blog post for details). Optuna has JournalFileBackend, a storage backend for various file systems. It can be used on NFS, allowing Optuna to scale to multiple nodes.

In Optuna v4.0, the API for JournalStorage has been reorganized, and JournalStorage is officially supported. This official support guarantees its backward compatibility from v4.0. For details on the API changes, please refer to the Optuna v4.0 Migration Guide. 

import optuna
from optuna.storages import JournalStorage
from optuna.storages.journal import JournalFileBackend

def objective(trial: optuna.Trial) -> float:
    ...

storage = JournalStorage(JournalFileBackend("./optuna_journal_storage.log"))
study = optuna.create_study(storage=storage)
study.optimize(objective)

Significant Speedup of Multi-Objective TPESampler

Before v4.0, the multi-objective TPESampler sometimes limits the number of trials during optimization due to the sampler bottleneck after a few hundred trials. Optuna v4.0 drastically improves the sampling speed, e.g., 300 times faster for three objectives with 200 trials, and enables users to handle much more trials. Please check the blog post for details.

Introduction of a New Terminator Algorithm

Optuna Terminator was originally introduced for hyperparameter optimization of machine learning algorithms using cross-validation. To accept broader use cases, Optuna v4.0 introduced the Expected Minimum Model Regret (EMMR) algorithm. Please refer to the EMMREvaluator document for details.

Enhancements of Constrained Optimization

We have gradually expanded the support for constrained optimization. In v4.0, study.best_trial and study.best_trials start to support constraint optimization. They are guaranteed to satisfy the constraints, which was not the case previously.

Breaking Changes

Optuna removes deprecated features in major releases. To prevent users' code from suddenly breaking, we take a long interval between when a feature is deprecated and when it is removed. By default, features are removed when the major version has increased by two since the feature was deprecated. For this reason, the main target features for removal in v4.0 were deprecated at v2.x. Please refer to the migration guide for the removed features list.

  • Delete deprecated three integrations, skopt, catalyst, and fastaiv1 (optuna/optuna-integration#114)
  • Remove deprecated CmaEsSampler from integration (optuna/optuna-integration#116)
  • Remove verbosity of LightGBMTuner (optuna/optuna-integration#136)
  • Move positional args of LightGBMTuner (optuna/optuna-integration#138)
  • Remove multi_objective (#5390)
  • Delete deprecated _ask and _tell (#5398)
  • Delete deprecated --direction(s) arguments in the ask command (#5405)
  • Delete deprecated three integrations, skopt, catalyst, and fastaiv1 (#5407)
  • Remove the default normalization of importance in f-ANOVA (#5411)
  • Remove samplers.intersection (#5414)
  • Drop implicit create-study in ask command (#5415)
  • Remove deprecated study optimize CLI command (#5416)
  • Remove deprecated CmaEsSampler from integration (#5417)
  • Support constrained optimization in best_trial (#5426)
  • Drop --study in cli.py (#5430)
  • Deprecate constraints_func in plot_pareto_front function (#5455)
  • Rename some class names related to JournalStorage (#5539)
  • Remove optuna.samplers.MOTPESampler (#5640)

New Features

Enhancements

  • Pass two arguments to the forward of ConstrainedMCObjective to support botorch=0.10.0 (optuna/optuna-integration#106)
  • Speed up non-dominated sort (#5302)
  • Make 2d hypervolume computation twice faster (#5303)
  • Reduce the time complexity of HSSP 2d from O(NK^2 log K) to O((N - K)K) (#5346)
  • Introduce lazy hypervolume calculations in HSSP for speedup (#5355)
  • Make plot_contour faster (#5369)
  • Speed up to_internal_repr in CategoricalDistribution (#5400)
  • Allow users to modify categorical distance more easily (#5404)
  • Speed up WFG by NumPy vectorization (#5424)
  • Check whether the study is multi-objective in sample_independent of GPSampler (#5428)
  • Suppress warnings from numpy in hypervolume computation (#5432)
  • Make an option to assume Pareto optimality in WFG (#5433)
  • Adapt multi objective to NumPy v2.0.0 (#5493)
  • Enhance the error message for integration installation (#5498)
  • Reduce journal size of JournalStorage (#5526)
  • Simplify a SQL query for getting the trial_id of best_trial (#5537)
  • Add import check for artifact store objects (#5565)
  • Refactor _is_categorical() in optuna/optuna/visualization (#5587, thanks @kAIto47802!)
  • Speed up WFG by using a fact that hypervolume calculation does not need (second or later) duplicated Pareto solutions (#5591)
  • Enhance the error message of multi_objective deletion (#5641)

Bug Fixes

  • Log None objective trials correctly (optuna/optuna-integration#119, thanks @neel04!)
  • Pass two arguments to the forward of ConstrainedMCObjective in qnei_candidates_func (optuna/optuna-integration#124, thanks @alxhslm!)
  • Allow single split cv in OptunaSearchCV (optuna/optuna-integration#128, thanks @sgerloff!)
  • Update BoTorch samplers to support new constraints interface (optuna/optuna-integration#132, thanks @alxhslm!)
  • Fix WilcoxonPruner bug when best_trial has no intermediate value (#5354)
  • Debug an error caused by convergence in GPSampler (#5359)
  • Fix average_is_best implementation in WilcoxonPruner (#5366)
  • Create an unique renaming filename for each release operation in lock systems of JournalStorage (#5389)
  • Fix _normalize_value for incomplete trials (#5422)
  • Fix heartbeat for race condition (#5431)
  • Guarantee weights_below to be finite in MOTPE (#5435)
  • Fix _log_complete_trial for constrained optimization (#5462)
  • Convert step to int in report (#5488)
  • Use a fixed seed value when seed=None in GridSampler (#5490)
  • Acquire session lock only for write in heartbeat (#5496)
  • Fix a bug in _create_new_trial and refactor it (#5497)
  • Add rdb create new trial test (#5525)
  • Fix a bug in sample_normalized_param for GPSampler (#5543)
  • Fix the error caused in plot_contour() with an impossible pair of variables (#5630, thanks @kAIto47802!)
  • Fix inappropriate behavior in plot_rank() when None values exist in trial (#5634, thanks @kAIto47802!)

Installation

Documentation

  • Enhance README.md (optuna/optuna-integration#126)
  • Add document page and fix docstring and comments (optuna/optuna-integration#134)
  • Update the docstring of PyCmaSampler (optuna/optuna-integration#145)
  • Add Dashboard and OptunaHub links to the document header (optuna/optuna-integration#155)
  • Remove duplicated license definition (optuna/optuna-integration#156)
  • Use sphinx rst syntax (#5345)
  • Fix typo (#5351)
  • Update docs about show_progress_bar (#5393)
  • Fix artifact tutorial (#5451)
  • Revise docs to specify ArtifactStore methods as non-public (#5474)
  • Improve the docstring of JournalFileStorage (#5475)
  • Improve make clean in docs (#5487)
  • Improve the example of chemical structures in optuna artifact tutorial (#5491)
  • Add FAQ for artifact store remove API (#5501)
  • Add a documentation for ArtifactMeta (#5511)
  • Change docs version from latest to stable (#5518)
  • Add a list of supported storage for artifact store (#5532)
  • Rename filenames for JournalFileStorage in documents and a tutorial (#5535)
  • Add explanation for lock_obj to JournalFileStorage docstring (#5540)
  • Fix storages document sections (#5553)
  • Add Returns to the docstring of load_study (#5554, thanks @kAIto47802!)
  • Fix paths related to storages.journal (#5560)
  • Rename filename for JournalFileBackend in examples (#5562)
  • Add thumbnail (#5575)
  • Add link to optunahub-registry (#5586)
  • Fix minor artifacts document issues (#5592)
  • Add the OptunaHub link to the document header (#5595)
  • Improve a docstring of CmaEsSampler (#5603)
  • Fix the expired links caused by the directory name change in optuna-examples (#5623, thanks @kAIto47802!)
  • Remove the use of evaluator.evaluate function in the example of PedAnovaImportanceEvaluator (#5632)
  • Add the news section to README.md (#5636)
  • Add SNS URLs to README.md (#5637)
  • Add TPE blog post link to doc (#5638)
  • Add artifact store post link (#5639)
  • Add installation guide for visualization tutorial (#5644, thanks @kAIto47802!)
  • Remove a broken link (#5648)
  • Update the News section of README.md (#5649)

Examples

Tests

  • Suppress ExperimentalWarnings (optuna/optuna-integration#108)
  • Add a unit test for convergence of acquisition function in GPSampler (#5365)
  • Remove unused block (#5368)
  • Implement backward compatibility tests for JournalStorage (#5486)
  • Add some tests for visualization functions (#5599)

Code Fixes

  • Align run name strategy between MLflowCallback and track_in_mlflow method (optuna/optuna-integration#111, thanks @TTRh!)
  • Use TYPE_CHECKING for ObjectiveFuncType (optuna/optuna-integration#113)
  • Ignore ExperimentalWarning for track_in_wandb (optuna/optuna-integration#122)
  • Remove unused module (optuna/optuna-integration#127)
  • Apply formatter and follow the Optuna conventions (optuna/optuna-integration#133)
  • Refactor an internal process of BoTorchSampler (optuna/optuna-integration#147)
  • Add missing experimental decorators to the comet integration (optuna/optuna-integration#148)
  • Use __future__.annotations (optuna/optuna-integration#150)
  • Fix E721 errors (optuna/optuna-integration#151)
  • Fix future annotations in percentile.py (#5322, thanks @aaravm!)
  • Delete examples directory (#5329)
  • Simplify annotations in optuna/pruners/_hyperband.py (#5338, thanks @keita-sa!)
  • Simplify annotations in optuna/pruners/_successive_halving.py and optuna/pruners/_threshold.py (#5343, thanks @keita-sa!)
  • Reduce test warnings (#5344)
  • Simplify type annotations for storages/_base.py (#5352, thanks @Obliquedbishop!)
  • Use TYPE_CHECKING for Study in samplers (#5391)
  • Refactor _normalize_objective_values in NSGA-III to supress RuntimeWarning (#5399)
  • Make ObjectiveFuncType available externally (#5401)
  • Replace numpy with np (#5412)
  • Bundle experimental feature warning (#5434)
  • Simplify annotations in _brute_force.py (#5441)
  • Add __future__.annotations to base sampler (#5442)
  • Introduce __future__.annotations to TPE-related modules (#5443)
  • Adapt to __future__.annotations in optuna/storages/_rdb/models.py (#5452, thanks @aisha-partha!)
  • Debug an unintended assertion error in GPSampler (#5484)
  • Make WFG a function (#5504)
  • Enhance the comments in create_new_trial (#5510)
  • Replace single trailing underscore with double trailing underscore in .rst files (#5514, thanks @47aamir!)
  • Fix hyperlinks to use double trailing underscores (#5515, thanks @virendrapatil24!)
  • Replace _ with __ in the link in Sphinx (#5517)
  • Refactor plot_parallel_coordinate() (#5527, thanks @karthikkurella!)
  • Remove an obsolete TODO comment (#5528)
  • Unite the argument order of artifact APIs (#5533)
  • Make assume_unique_lexsorted in is_pareto_front required (#5534)
  • Simplify annotations in terminator module (#5536)
  • Simplify the type annotations in samplers/_qmc.py (#5538, thanks @kAIto47802!)
  • Rename solution_set to loss_vals in hypervolume computation (#5541)
  • Expand the type of callbacks in optimize to Iterable (#5542, thanks @kz4killua!)
  • Organize the directory structure of JournalStorage (#5544)
  • Add followup of journal storage module structure organization PR (#5546)
  • Move RetryFailedTrialCallback to optuna.storages._callbacks (#5551)
  • Change removed_version of deprecated JournalStorage classes (#5552)
  • Reorganize a journal storage module structure (#5555)
  • Remove unused private deprecated class BaseJournalLogSnapshot (#5561)
  • Replace relative import to absolute path (#5569, thanks @RektPunk!)
  • Simplify annotation in importance (#5578, thanks @RektPunk!)
  • Simplify annotation in trial (#5579, thanks @RektPunk!)
  • Fix mypy warnings in matplotlib/_contour.py with Python3.11 (#5580)
  • Remove redundant loop in _get_rank_subplot_info (#5581, thanks @RektPunk!)
  • Simplify annotations in _partial_fixed.py (#5583)
  • Simplify annotation in _cmaes.py (#5584, thanks @RektPunk!)
  • Simplify annotations related to GridSampler (#5588)
  • Reorganize names related to search_space in terminator/improvement/evaluator.py (#5594)
  • Refactor _bisect in _truncnorm.py (#5598)
  • Simplify annotation storages (#5608, thanks @RektPunk!)
  • Add a note about np.unique (#5615)
  • Fix ruff errors except for E731 (#5627)

Continuous Integration

Other

Thanks to All the Contributors and Sponsors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@47aamir, @Alnusjaponica, @HideakiImamura, @Obliquedbishop, @RektPunk, @TTRh, @aaravm, @aisha-partha, @alxhslm, @c-bata, @caleb-kaiser, @contramundum53, @eukaryo, @gen740, @kAIto47802, @karthikkurella, @keisuke-umezawa, @keita-sa, @kz4killua, @nabenabe0928, @neel04, @not522, @nzw0301, @porink0424, @sgerloff, @toshihikoyanase, @virendrapatil24, @y0z

Optuna is sponsored by the following sponsors on GitHub.

@AlphaImpact, @dec1costello, @dubovikmaster, @shu65, @raquelhortab

Check out latest releases or
releases around optuna 4.0.0

Don't miss a new optuna release

NewReleases is sending notifications on new releases.

Get notifications