pypi accelerate 1.16.0
v1.16.0: broader accelerator support, launcher and FSDP improvements

47 minutes ago

Accelerator support

Accelerate now uses torch.accelerator more broadly to support registered PyTorch backends without requiring a separate device-specific branch throughout the library. Existing integrations retain their native APIs where needed for compatibility.

This also includes initial improvements for single-chip TPU usage through torch-tpu. Broader TPU support remains in progress.

  • Simplify backend discovery and extend custom accelerator support — @SunMarc (#4335, #4547).
  • Fix tied weights, safetensors loading, memory reporting, and FP16 validation for single-chip torch-tpu usage — @tengomucho (#4570).

Launcher improvements

accelerate launch now exits cleanly when interrupted with Ctrl+C or SIGTERM, cleaning up child workers. Distributed launches also handle torchrun shutdown signals without an extra launcher traceback.

  • Clean up child processes on interruption and improve --quiet and --debug behavior — @SunMarc (#4573).
  • Surface child-process errors in simple_launcher while preserving live output — @albertvillanova (#4305).

FSDP

Several fixes address RAM-efficient loading, including compatibility with Transformers loading models onto the meta device on nonzero ranks. FSDP1 now reconnects optimizer references when initialization replaces meta parameters, and FSDP2 correctly distinguishes those parameters when mapping them to their sharded replacements.

FSDP2 also supports preparing a model without an optimizer, allowing workflows such as Trainer.evaluate() before training.

🚨🚨🚨 ## Requirements and removals

Accelerate now requires Python 3.11+ and PyTorch 2.6+. This release also removes several deprecated integrations and APIs.

  • Raise the minimum Python version to 3.11 and PyTorch version to 2.6 — @SunMarc (#4459, #4461).
  • Remove the PiPPy and MS-AMP integrations — @SunMarc (#4462, #4463).
  • Remove the deprecated FP8RecipeKwargs. Use AORecipeKwargs for TorchAO or TERecipeKwargs for Transformer Engine — @SunMarc (#4464).

Minor fixes

  • Propagate notebook_launcher mixed precision to worker processes — @simpleqt (#4331).
  • Stop accelerate launch from forcing torch.compile(dynamic=False) — @yupengtang (#4254).
  • Avoid importing torch.distributed in model_has_dtensor on builds without distributed support — @shoemoney (#4250).
  • Avoid upcasting placeholder weights on non-main ranks during RAM-efficient loading — @pagel-s (#4369).
  • Restore FSDP2 FP16 GradScaler state without advancing it — @HaomingSong (#4291).
  • Preserve output dtypes under FSDP2 mixed precision, including losses computed in FP32 — @SunMarc (#4403).
  • Skip _no_split_modules classes absent from the model when configuring auto-wrapping — @qgallouedec (#4384).
  • Support clip_grad_value_ under FSDP2 — @hjinnkim (#4386).
  • Allow tensor parallelism alongside replicated data parallelism — @qgallouedec (#4193).
  • Use Transformers’ model capability checks for context parallelism — @qgallouedec (#4198).
  • Compile checkpointed blocks across graph breaks in compile_regions — @qgallouedec (#4524).
  • Add DeepSpeed sequence-parallel configuration options — @eilamc14 (#4565).
  • Preserve an explicit zero gradient-clipping value in DeepSpeed configuration — @Titus-von-Koeller (#4406).
  • Align Accelerator.clip_grad_norm_ with Transformers — @michaelbenayoun (#4266).
  • Prefer all_gather_single over the deprecated all_gather_into_tensor — @Yi-111-a (#4272).
  • Fix wait_for_everyone() in CPU distributed mode on machines with an accelerator — @JoeyTan21 (#4354).
  • Fix a VRAM leak from cached INT8 weights during CPU or disk offload — @edmcman (#4325).
  • Fix checkpoint loading with modules offloaded to disk — @yupengtang (#4352).
  • Reduce key-merging overhead in OffloadedWeightsLoader — @emme1t (#4281).
  • Fix memory-size calculations for sub-byte and packed dtypes — @VihaanAgarwal (#4237).
  • Fix balanced_low_0 looking up memory limits by position instead of device ID — @Om-singhaI (#4298).
  • Fix an IndexError when a tied parameter’s partner module is split — @simpleqt (#4344).
  • Preserve hooks when calling unwrap_model(keep_fp32_wrapper=False) — @yupengtang (#4255).
  • Restrict compiler disabling in AlignDevicesHook to the offload path — @jiqing-feng (#4327).
  • Fix shape metadata when copying tensors across devices — @MrCapricornLiu (#4245).
  • Preserve dataloader options when using skip_first_batches, including drop_last, nonblocking transfers, the slicing function, and the device mesh — @truecallerabreham (#4296).
  • Fix metric padding removal after skip_first_batches — @Excelius-Wang (#4235).
  • Fix split_between_processes on tuples with apply_padding=True — @shoutoutuoadi325 (#4307).
  • Fix next-token labels and gradient-accumulation target weighting in the autoregressive training example — @Titus-von-Koeller (#4303).
  • Add logging verbosity helpers — @SunMarc (#4556).
  • Read ACCELERATE_BYPASS_DEVICE_MAP case-insensitively — @Sreekant13 (#4241).
  • Fix ParallelismConfig.to_json() returning None — @Linxiushen (#4326).

New contributors

Thanks to everyone making their first contribution!

Full changelog: v1.15.0 → v1.16.0

Don't miss a new accelerate release

NewReleases is sending notifications on new releases.