microsoft/nni v3.0

Release 3.0 -21/8/2023

on GitHub

Web Portal

• New look and feel

Neural Architecture Search

• Breaking change: nni.retiarii is no longer maintained and tested. Please migrate to nni.nas.
  • Inherit nni.nas.nn.pytorch.ModelSpace, rather than use @model_wrapper.
  • Use nni.choice, rather than nni.nas.nn.pytorch.ValueChoice.
  • Use nni.nas.experiment.NasExperiment and NasExperimentConfig, rather than RetiariiExperiment.
  • Use nni.nas.model_context, rather than nni.nas.fixed_arch.
  • Please refer to quickstart for more changes.
• A refreshed experience to construct model space.
  • Enhanced debuggability via freeze() and simplify() APIs.
  • Enhanced expressiveness with nni.choice, nni.uniform, nni.normal and etc.
  • Enhanced experience of customization with MutableModule, ModelSpace and ParamterizedModule.
  • Search space with constraints is now supported.
• Improved robustness and stability of strategies.
  • Supported search space types are now enriched for PolicyBaseRL, ENAS and Proxyless.
  • Each step of one-shot strategies can be executed alone: model mutation, evaluator mutation and training.
  • Most multi-trial strategies now supports specifying seed for reproducibility.
  • Performance of strategies have been verified on a set of benchmarks.
• Strategy/engine middleware.
  • Filtering, replicating, deduplicating or retrying models submitted by any strategy.
  • Merging or transforming models before executing (e.g., CGO).
  • Arbitrarily-long chains of middlewares.
• New execution engine.
  • Improved debuggability via SequentialExecutionEngine: trials can run in a single process and breakpoints are effective.
  • The old execution engine is now decomposed into execution engine and model format.
  • Enhanced extensibility of execution engines.
• NAS profiler and hardware-aware NAS.
  • New profilers profile a model space, and quickly compute a profiling result for a sampled architecture or a distribution of architectures (FlopsProfiler, NumParamsProfiler and NnMeterProfiler are officially supported).
  • Assemble profiler with arbitrary strategies, including both multi-trial and one-shot.
  • Profiler are extensible. Strategies can be assembled with arbitrary customized profilers.

Model Compression

• Compression framework is refactored, new framework import path is nni.contrib.compression.
  • Configure keys are refactored, support more detailed compression configurations. view doc
  • Support multi compression methods fusion.
  • Support distillation as a basic compression component.
  • Support more compression targets, like input, ouptut and any registered paramters.
  • Support compressing any module type by customizing module settings.
• Model compression support in DeepSpeed mode.
• Fix example bugs.
• Pruning
  • Pruner interfaces have fine-tuned for easy to use. view doc
  • Support configuring granularity in pruners. view doc
  • Support different mask ways, multiply zero or add a large negative value.
  • Support manully setting dependency group and global group. view doc
  • A new powerful pruning speedup is released, applicability and robustness have been greatly improved. view doc
  • The end to end transformer compression tutorial has been updated, achieved more extreme compression performance. view doc
  • Fix config list in the examples.
• Quantization
  • Support using Evaluator to handle training/inferencing.
  • Support more module fusion combinations. view doc
  • Support configuring granularity in quantizers. view doc
  • Bias correction is supported in the Post Training Quantization algorithm.
  • LSQ+ quantization algorithm is supported.
• Distillation
  • DynamicLayerwiseDistiller and Adaptive1dLayerwiseDistiller are supported.
• Compression documents now updated for the new framework, the old version please view v2.10 doc.
• New compression examples are under nni/examples/compression
  • Create a evaluator: nni/examples/compression/evaluator
  • Pruning a model: nni/examples/compression/pruning
  • Quantize a model: nni/examples/compression/quantization
  • Fusion compression: nni/examples/compression/fusion

Training Services

• Breaking change: NNI v3.0 cannot resume experiments created by NNI v2.x
• Local training service:
  • Reduced latency of creating trials
  • Fixed "GPU metric not found"
  • Fixed bugs about resuming trials
• Remote training service:
  • reuse_mode now defaults to False; setting it to True will fallback to v2.x remote training service
  • Reduced latency of creating trials
  • Fixed "GPU metric not found"
  • Fixed bugs about resuming trials
  • Supported viewing trial logs on the web portal
  • Supported automatic recover after temporary server failure (network fluctuation, out of memory, etc)
• Get rid of IoC and remove unused training services.