hydra-ecosystem/hydra v1.4.0rc1

Hydra 1.4.0rc1

on GitHub

Hydra 1.4.0rc1 is a prerelease. Please test your applications and report regressions.

Hydra 1.4 is the first new feature release line since Hydra 1.3 in December 2022. It builds on OmegaConf 2.4, which is also the first feature release since 2022. See the OmegaConf 2.4 release notes.

Hydra supports Python 3.10 through 3.14. Upgrade bundled plugins alongside Hydra; the 1.4 plugins require Hydra Core >=1.4.0.dev1,<1.5.0.dev0.

Review the Hydra 1.3 to 1.4 migration guide before upgrading.

Source commit: ee782f78a3.

Known Ray limitations

End-to-end AWS launcher behavior has not been validated for this release. AWS integration tests remain disabled, and no new test AMI was built. The full integration run with exact-minimum Ray 2.55.0 remains incomplete; hosted validation used dependencies within the supported ranges.

Hydra core

Features

• Add explicit ConfigStore replacement and warn on implicit collisions.
  Avoid repeated config registration during plugin discovery. (#2453)
• Honor _recursive_ on non-target config nodes during instantiation. (#2577)
• Add the Hydra Landscape for discovering projects that use and extend Hydra. (#3206)
• hydra.utils.instantiate() calls on OmegaConf inputs without call-site
  overrides no longer make an additional full-tree copy. Benchmarked calls run 4
  to 10 times faster. (#3216)
• Add explicit override aliases for the CLI and Compose API. (#3224)
• Add a Compose initializer for ordered config sources and caller-relative paths. (#3230)
• Recompose experimental reruns from saved overrides using normal job execution. (#3248)
• Support bracket paths and escaped characters in config value override keys. (#3250)
• Discover plugins through entry points while retaining legacy namespace support. (#3252)
• Add instantiate support for OmegaConf tuple configurations. (#3287)
• Preserve user frames and chained causes in application tracebacks.
  Mark hidden Hydra frames after the application starts.
  Report nontransportable remote exceptions as RuntimeError with their type named. (#3455)
• Allow absolute config paths in the Compose API initialize() method. (#3485)

API Change (Renames, deprecations and removals)

• Remove assert_regex_match in favor of assert_multiline_regex_search. (#1749)
• Remove the unsupported Torchrun launcher and the contrib plugin area. (#2119)
• A dict or DictConfig call-site argument to hydra.utils.instantiate() now
  replaces a configured plain, non-target mapping passed to a target instead of
  merging into it. (#2350)
• instantiate() now passes dataclass and attrs instances supplied as call-site
  arguments through unchanged instead of interpreting them as Structured Configs.
  Use OmegaConf.structured(instance) when an instance should still be merged and
  recursively instantiated as configuration. See the
  Hydra 1.4 migration guide
  for details and examples. (#2364)
• Deprecate hydra.job.override_dirname in favor of hydra_override_dirname. (#3018)
• Remove the experimental on_compose_config callback hook. (#3023)
• Certain modules that may introduce a security vulnerability can no longer be instantiated by default. This is not exhaustive, do not rely on this to prevent configs from executing dangerous code. (#3074)
• Drop Python 3.7, 3.8, and 3.9 support; Hydra now supports Python 3.10 through 3.14. This consolidates the previous Python support update from #2854. (#3154)
• Reject parent traversal in all Defaults List config paths. (#3202)
• hydra.utils.instantiate() now resolves configured interpolations as recursive
  traversal reaches them instead of resolving the entire input configuration up
  front. With _recursive_=False, _convert_="none", and no call-site overrides,
  configured OmegaConf containers passed as target arguments retain their
  identity. The source configuration is temporarily read-only during instantiation;
  constructors can opt in to mutation with OmegaConf's read_write() context
  manager. Such mutations affect the input configuration. See the
  Hydra 1.4 instantiate migration guide
  for details.
  Plain Python call-site overrides must be concrete runtime values; ??? and
  interpolation syntax are rejected. Explicit OmegaConf containers retain normal
  OmegaConf semantics. (#3216)
• Normalize Defaults List items containing a slash in their selected value (e.g. foo: bar/baz -> foo/bar: baz) early during Defaults List creation. (#3222)
• Deprecate version_base in preparation for its removal in Hydra 1.5. Require
  Hydra 1.4 or newer for hydra-configen. (#3237)
• Remove PickleJobInfoCallback; experimental rerun now takes a job directory. (#3248)
• Deprecate config-selected target resolution without an execution whitelist. (#3259)
• Instantiate launcher and sweeper plugin configs non-recursively. (#3268)
• Breaking change: Native tuples now create TupleConfig values instead of ListConfig values, and conversion returns native tuples instead of lists. Code that expects tuple-derived configs to be ListConfig, checks OmegaConf.is_list(), or expects converted values to be list must be updated. See Migrating tuple usage in OmegaConf 2.4. (#3287)
• Hydra no longer suggests HYDRA_FULL_ERROR=1 after a sanitized error. The variable still
  disables traceback sanitization and is now documented in the developer guide as a framework
  debugging facility. (#3329)
• Remove Hydra 1.1 compatibility behavior and version_base="1.1" support. (#3345)
• Remove Hydra 1.2 migration behavior and version_base="1.2" support. (#3346)
• Propagate target exceptions without wrapping them in InstantiationException.
  Make InstantiationException no longer inherit from CompactHydraException.
  Add full_key notes to nested target failures on Python 3.11 and newer.
  Stop wrapping failures while constructing deferred targets.
  Preserve string exception notes across launcher transport.
  Preserve plugin constructor ImportError messages instead of missing-plugin hints. (#3361)
• Reject backslashes in Defaults List config group and option paths; '/' is the only supported config group separator. (#3362)
• Deprecate LogJobReturnCallback as a no-op; Hydra logs job failures directly. (#3401)
• Make source configs temporarily read-only during copy-free instantiation.
  Reject pickling Hydra _partial_ factories. (#3436)
• Range sweep casts now convert each generated value during iteration. (#3520)

Bug Fixes

• Honor custom sys.excepthook handlers when rendering sanitized task exceptions. (#1431)
• Allow Defaults List interpolation to reference relocated config groups. Unsupported OmegaConf resolvers in Defaults Lists are no longer accepted. (#1855)
• Allow nested Defaults List interpolations to use outer config-group choices. (#1899)
• Make glob sweeps respect the primary config search path. (#1942)
• Reject unsupported keywords in Defaults List entries. (#1951)
• ConfigStore.store with an empty string group now stores the config at the repository root, exactly like group=None, instead of creating an empty-named config group that makes --help generation recurse without terminating. (#2121)
• Show application help when a mandatory Defaults List choice is unresolved. (#2170)
• Report misplaced overrides followed by list-valued defaults clearly. (#2236)
• Load configs from regular Python packages without requiring init.py. (#2412)
• Fix TypeError when deleting a ListConfig item by index via CLI override (e.g. ~config.x.0) (#2477)
• Preserve nested dataclass targets during instantiation. (#2507)
• Fix a command line parsing bug in multirun mode when PYTHONOPTIMIZE=1 (#2570)
• Preserve tracebacks for transportable job exceptions across remote launchers. (#2664)
• Resolve hydra.job.env_set interpolations before setting environment variables. (#2800)
• Support deleting config keys with null or missing values. (#2829)
• Support deleting non-overridable config-path defaults from the CLI. (#2843)
• Fix command-line defaults appends from nested primary configs. (#2875)
• Report unsupported .yml config files with a clear error. (#2889)
• Fix StructuredConfigStore using old singleton copy when using lazy imports (#2928)
• Fix unexpected resolution side-effect that caused modifications to the input config parent in hydra.utils.instantiate (#3001)
• Fixed defaults list interpolation inside config groups. (#3036)
• Allow overrides on both sides of Hydra command-line flags. (#3053)
• Fix on_job_end and on_run_end callbacks not being called when a job is stopped with a KeyboardInterrupt. (#3078)
• Log multirun failures in each job's log file. (#3215)
• Fix sort() of a range() returning wrong values when the range does not land exactly on stop (e.g. sort(range(0, 5, 2))). (#3228)
• Make a Defaults List override replace an earlier override of the same config group, so that the last override in depth first order wins. (#3229)
• Reject swept configs that change multirun launcher or sweeper configuration. (#3231)
• Fix bash completion to support script paths containing hyphens (e.g. python my-app/run.py). (#3243)
• Harden instantiate target blacklist coverage for security-sensitive callables. (#3259)
• Prevent Bash completion from executing non-Hydra Python scripts. (#3275)
• A Defaults List override that does not target an earlier Group Default now fails to compose even when a command line append introduces a matching group, instead of silently discarding the appended value. (#3318)
• Make configured interpolations in hydra.utils.instantiate() resolve against
  the complete set of call-site overrides without modifying the input
  configuration. (#3356)
• Reject Defaults List entries that combine optional and override. (#3405)
• instantiate() blocks unsafe targets and refuses to whitelist
  config-controlled execution surfaces, including aliases, callable wrappers,
  deferred dispatch, and unauthorized callable results.
  Hydra logging configuration now applies the same execution policy to dynamically
  resolved handlers, formatters, filters, queues, listeners, and external values.
  Brace-style logging formatters and descriptor callback wrappers are also rejected.
  Invalid configured formatter results are ignored with a warning. (#3411)
• Install HydraConfig for the multirun controller before callbacks and the sweeper are initialized. (#3427)
• Preserve merged ancestor values when instantiating context-parented subtrees. (#3436)
• Prevent configuration from accessing or modifying Hydra execution-policy state.
  Validate policy integrity before resolving configured targets.
  Prevent declarative configuration from mutating the process environment. (#3442)
• Show config origins in Defaults List errors and keep malformed-list errors concise. (#3454)
• Preserve sweep tags and shuffle settings when casting values. (#3456)
• Preserve OmegaConf literals like \??? in composition and instantiation. (#3480)
• Call paired callback end hooks when jobs or sweeps exit with errors. (#3491)
• Sorting a shuffled sweep now clears shuffling and yields ordered values. (#3520)

Configuration structure changes

• Group-derived hydra.mode no longer selects a run mode and may cause an error. (#3440)

Improved Documentation

• Add a reference to Hydra's special fields and directives. (#1622)
• Fix broken Binder link in the Jupyter notebooks documentation page. (#2149)
• Clarify multi-select config group documentation and example. (#2695)
• Clarify config search path precedence and how to override a primary source. (#2758)
• Document how to prepare for Hydra 1.4 and track current breaking changes. (#3351)
• Adopt a project-controlled Code of Conduct and reporting address. (#3394)
• Document environment-specific overrides for Hydra's default configuration. (#3406)
• Add a reference page for @hydra.main() behavior and parameters. (#845)

Maintenance Changes

• Remove missing _self_ warnings from example configurations. (#1781)
• Use OmegaConf's vendored version of antlr4 runtime to prevent conflicts with other dependencies. See hydra-ecosystem/omegaconf#1114 for more details. (#2699)
• Remove packaging from Hydra's runtime dependencies. (#3290)
• Move Hydra to the Hydra Ecosystem GitHub organization. Update project links and
  contribution guidance following the repository transfer. (#3372)
• Avoid reparsing command-line overrides during run mode selection and composition. (#3440)
• Include the ANTLR license with source distributions containing its build JAR. (#3444)
• Modernize public type annotations with Python 3.10 syntax. (#3486)

Ax Sweeper

Features

• Support returning a dict (rather than just a scalar value) from the @hydra.main-decorated task function (#2330)

API Change (Renames, deprecations and removals)

• Require Hydra Core 1.4 for the Ax sweeper. (#3324)
• End first-party Ax Sweeper releases after Hydra 1.4; remove code in 1.5. (#3506)

Maintenance Changes

• Update Ax Sweeper to Ax 1.2 and support Python 3.11 through 3.14. (#3176)

Colorlog

Features

• Support python3.11 (#2443)

API Change (Renames, deprecations and removals)

• Require Hydra Core 1.4 for the Colorlog plugin. (#3324)

Bug Fixes

• Fix log file path for colorlog to work correctly if Hydra does not change the current working directory (#2663)

Joblib Launcher

Features

• Restore Joblib multiprocessing with per-job process isolation. (#2187)
• Support python3.11 (#2443)
• Support configuring Joblib worker inner thread limits. (#3185)

API Change (Renames, deprecations and removals)

• Require Hydra Core 1.4 and Joblib 1.5.3. (#3323)

Bug Fixes

• Fix the Joblib Launcher example app config path. (#3184)

Nevergrad Sweeper

Features

• Support python3.11 (#2443)

API Change (Renames, deprecations and removals)

• Require Hydra Core 1.4 for the Nevergrad sweeper. (#3324)

Configuration structure changes

• Add params and deprecate parametrization in the Nevergrad Sweeper. (#1890)

Maintenance Changes

• Update the minimum Nevergrad Sweeper dependency to Nevergrad 1.0.12. (#3186)

Optuna Sweeper

Features

• Add fault tolerance via max_failure_rate parameter (#1513)
• Support python3.11 (#2443)
• Support Optuna 4.9.+, add GP/QMC samplers, and remove MOTPE support (#3211)
• Align Optuna sampler configs with Optuna 4.9 defaults and options. (#3425)
• Add NSGA-III sampler support to the Optuna Sweeper. (#3426)
• Add Optuna v5 support and configurable NSGA-II and NSGA-III mutations. (#3443)

API Change (Renames, deprecations and removals)

• Require Hydra Core 1.4 for the Optuna sweeper. (#3324)
• Remove the deprecated hydra.sweeper.search_space configuration. (#3346)
• Cast ranges use categorical choices; grids include all eligible values. (#3521)

Bug Fixes

• Continue Optuna sweeps after individual NaN or invalid trial results (#2237)
• Resolve custom search-space callbacks through instantiate(). (#3411)
• Preserve range sweep values and reject empty ranges and zero steps. (#3521)

Ray Launcher

API Change (Renames, deprecations and removals)

• Require Hydra Core 1.4 for the Ray launcher. (#3324)
• Require Ray >=2.55.0,<3 and cloudpickle >=3.1.2,<4. (#3528)

Bug Fixes

• Avoid default object_store_memory when connecting to an existing Ray cluster (#2893)

RQ Launcher

Features

• Support python3.11 (#2443)

API Change (Renames, deprecations and removals)

• Require Hydra Core 1.4 for the RQ launcher. (#3324)

Bug Fixes

• Upgrade RQ launcher dependencies and validate worker result serialization. (#2338)

Submitit Launcher

Features

• Allow tasks_per_node to be omitted from Submitit Slurm submissions. (#2157)
• Add support to Hydra's slurm launcher config for submitit's srun_args parameter. (#2429)
• Support python3.11 (#2443)
• Allow disabling srun in the Slurm launcher; require Submitit 1.5.0 or newer. (#2814)
• Add support for selecting the Python executable in the Submitit launcher. (#3309)

API Change (Renames, deprecations and removals)

• Require Hydra Core 1.4 for the Submitit launcher. (#3324)