Accelerator support
Accelerate now uses torch.accelerator more broadly to support registered PyTorch backends without requiring a separate device-specific branch throughout the library. Existing integrations retain their native APIs where needed for compatibility.
This also includes initial improvements for single-chip TPU usage through torch-tpu. Broader TPU support remains in progress.
- Simplify backend discovery and extend custom accelerator support — @SunMarc (#4335, #4547).
- Fix tied weights, safetensors loading, memory reporting, and FP16 validation for single-chip
torch-tpuusage — @tengomucho (#4570).
Launcher improvements
accelerate launch now exits cleanly when interrupted with Ctrl+C or SIGTERM, cleaning up child workers. Distributed launches also handle torchrun shutdown signals without an extra launcher traceback.
- Clean up child processes on interruption and improve
--quietand--debugbehavior — @SunMarc (#4573). - Surface child-process errors in
simple_launcherwhile preserving live output — @albertvillanova (#4305).
FSDP
Several fixes address RAM-efficient loading, including compatibility with Transformers loading models onto the meta device on nonzero ranks. FSDP1 now reconnects optimizer references when initialization replaces meta parameters, and FSDP2 correctly distinguishes those parameters when mapping them to their sharded replacements.
FSDP2 also supports preparing a model without an optimizer, allowing workflows such as Trainer.evaluate() before training.
- Fix FSDP meta-parameter initialization and optimizer mapping — @SunMarc (#4576).
- Fix FSDP2 RAM-efficient loading of ignored parameters and buffers — @SunMarc (#4568).
- Fix FSDP2 sharding when activation checkpointing is enabled — @SunMarc @Titus-von-Koeller (#4575).
- Allow preparing a model without an optimizer under FSDP2 — @qgallouedec (#4519).
🚨🚨🚨 ## Requirements and removals
Accelerate now requires Python 3.11+ and PyTorch 2.6+. This release also removes several deprecated integrations and APIs.
- Raise the minimum Python version to 3.11 and PyTorch version to 2.6 — @SunMarc (#4459, #4461).
- Remove the PiPPy and MS-AMP integrations — @SunMarc (#4462, #4463).
- Remove the deprecated
FP8RecipeKwargs. UseAORecipeKwargsfor TorchAO orTERecipeKwargsfor Transformer Engine — @SunMarc (#4464).
Minor fixes
- Propagate
notebook_launchermixed precision to worker processes — @simpleqt (#4331). - Stop
accelerate launchfrom forcingtorch.compile(dynamic=False)— @yupengtang (#4254). - Avoid importing
torch.distributedinmodel_has_dtensoron builds without distributed support — @shoemoney (#4250). - Avoid upcasting placeholder weights on non-main ranks during RAM-efficient loading — @pagel-s (#4369).
- Restore FSDP2 FP16 GradScaler state without advancing it — @HaomingSong (#4291).
- Preserve output dtypes under FSDP2 mixed precision, including losses computed in FP32 — @SunMarc (#4403).
- Skip
_no_split_modulesclasses absent from the model when configuring auto-wrapping — @qgallouedec (#4384). - Support
clip_grad_value_under FSDP2 — @hjinnkim (#4386). - Allow tensor parallelism alongside replicated data parallelism — @qgallouedec (#4193).
- Use Transformers’ model capability checks for context parallelism — @qgallouedec (#4198).
- Compile checkpointed blocks across graph breaks in
compile_regions— @qgallouedec (#4524). - Add DeepSpeed sequence-parallel configuration options — @eilamc14 (#4565).
- Preserve an explicit zero gradient-clipping value in DeepSpeed configuration — @Titus-von-Koeller (#4406).
- Align
Accelerator.clip_grad_norm_with Transformers — @michaelbenayoun (#4266). - Prefer
all_gather_singleover the deprecatedall_gather_into_tensor— @Yi-111-a (#4272). - Fix
wait_for_everyone()in CPU distributed mode on machines with an accelerator — @JoeyTan21 (#4354). - Fix a VRAM leak from cached INT8 weights during CPU or disk offload — @edmcman (#4325).
- Fix checkpoint loading with modules offloaded to disk — @yupengtang (#4352).
- Reduce key-merging overhead in
OffloadedWeightsLoader— @emme1t (#4281). - Fix memory-size calculations for sub-byte and packed dtypes — @VihaanAgarwal (#4237).
- Fix
balanced_low_0looking up memory limits by position instead of device ID — @Om-singhaI (#4298). - Fix an
IndexErrorwhen a tied parameter’s partner module is split — @simpleqt (#4344). - Preserve hooks when calling
unwrap_model(keep_fp32_wrapper=False)— @yupengtang (#4255). - Restrict compiler disabling in
AlignDevicesHookto the offload path — @jiqing-feng (#4327). - Fix shape metadata when copying tensors across devices — @MrCapricornLiu (#4245).
- Preserve dataloader options when using
skip_first_batches, includingdrop_last, nonblocking transfers, the slicing function, and the device mesh — @truecallerabreham (#4296). - Fix metric padding removal after
skip_first_batches— @Excelius-Wang (#4235). - Fix
split_between_processeson tuples withapply_padding=True— @shoutoutuoadi325 (#4307). - Fix next-token labels and gradient-accumulation target weighting in the autoregressive training example — @Titus-von-Koeller (#4303).
- Add logging verbosity helpers — @SunMarc (#4556).
- Read
ACCELERATE_BYPASS_DEVICE_MAPcase-insensitively — @Sreekant13 (#4241). - Fix
ParallelismConfig.to_json()returningNone— @Linxiushen (#4326).
New contributors
Thanks to everyone making their first contribution!
- @albertvillanova — #4247
- @Titus-von-Koeller — #4302
- @edmcman — #4325
- @emme1t — #4281
- @VihaanAgarwal — #4237
- @Om-singhaI — #4298
- @shoutoutuoadi325 — #4307
- @Yi-111-a — #4272
- @Linxiushen — #4326
- @truecallerabreham — #4296
- @pratikgx — #4348
- @JoeyTan21 — #4354
- @Excelius-Wang — #4235
- @pagel-s — #4369
- @MrCapricornLiu — #4245
- @eilamc14 — #4565