What's Changed
- Validate warmup_type in WarmupCosineLR like WarmupLR by @sohumt123 in #8151
- Run pull request code only inside a Modal Sandbox by @tohtana in #8170
- [AutoTP] Allow ZeRO stage 3 inference with tensor parallelism by @delock in #8167
- Fix autotuning get_val_by_key to search all nested subdicts by @ebarkhordar in #8177
- Enable support for Tutel when k != 1 for shared moe by @jinyouzhi in #8174
- NVMe write warning by @sfc-gh-truwase in #8164
- Guard LRRangeTest and OneCycle schedulers against zero step sizes by @ebarkhordar in #8166
- Fix WarmupLR collapsing multi-group base LRs to group 0's by @ebarkhordar in #8171
- Stabilize fork-sensitive nightly tests and AutoSP coverage by @tohtana in #8172
- Make the OneCycle stair counts actually change the schedule by @vineethsaivs in #8179
- [AutoTP] Enable HF
colwise_gather_outputto supportlm_headreplace by @jinyouzhi in #8146 - Scope DeepCompile compiler state to graph and engine lifecycles by @tohtana in #8159
- Clarify merge commit exemption for signoff check in AGENTS.md and CLAUDE.md by @jinyouzhi in #8194
- [AutoTP] Preserve universal checkpoint metadata for HuggingFace tp_plan by @jinyouzhi in #8187
- Derive AutoEP rank splits from the per-expert count exchange by @yh0903 in #8190
- Warn when zero.Init silently falls back to a single rank (#8084) by @akshansh47 in #8089
- Release GIL in blocking AIO bindings by @tohtana in #8196
- Enable ZeRO-3 linear wrapper for existing models by @tohtana in #8189
- Skip CUDA transformer forward tests when TransformerBuilder is incompatible by @sfc-gh-truwase in #8200
- Clamp capacity to num_tokens in MoE gating drop branches (complete #5353) by @ebarkhordar in #8155
- Preserve tensor learning rates across scheduler updates by @n33levo in #8202
- [AutoTP] Fix ZeRO-3 checkpoint consolidation to gather across TP and DP by @delock in #8168
- Add managed_gradient_accumulation for ZeRO stage 0/1 by @sfc-gh-truwase in #8184
- MoE dispatch: fix silent Tutel + TP corruption, speed up native and Tutel paths by @jinyouzhi in #8195
- Add keyword argument support to activation checkpointing by @zupengwang in #8182
- Remove redundant DCO workflow by @tohtana in #8188
- Preserve tensor learning rates in OneCycle by @tohtana in #8205
- Add Triton grouped-GEMM for MoE experts on Ampere/Ada by @hwchen2017 in #8180
- Wiring up offload_opt_states by @pengdurice in #8186
- Fix DeepCompile ZeRO-3 gathered parameter ownership by @tohtana in #8157
- Consolidate LR scheduler tensor-LR tests into a contract suite by @delock in #8206
- Fix MLU_Accelerator conformance to the DeepSpeedAccelerator ABC by @ebarkhordar in #8208
- Reduce redundant work in AutoEP token routing by @hwchen2017 in #8209
- Make PipelineModule.set_checkpoint_interval actually change the interval by @vineethsaivs in #8178
- Share DeepNVMe pinned-tensor manager and route swap buffers through I/O handles by @sfc-gh-truwase in #8212
- Stop trim_mean sorting the caller's list in place by @vineethsaivs in #8199
- Unmanaged gradient accumulation: ZeRO stage 2 support by @sfc-gh-truwase in #8203
- Fix ZeRO-3 all_reduce param fetch stride for padded parameters by @ebarkhordar in #8158
- Unmanaged gradient accumulation: ZeRO stage 3 support by @sfc-gh-truwase in #8217
- Update version.txt pre-0.19.4 release by @loadams in #8218
New Contributors
- @sohumt123 made their first contribution in #8151
- @ebarkhordar made their first contribution in #8177
- @yh0903 made their first contribution in #8190
- @akshansh47 made their first contribution in #8089
- @n33levo made their first contribution in #8202
- @zupengwang made their first contribution in #8182
- @pengdurice made their first contribution in #8186
Full Changelog: v0.19.3...v0.19.4