What's Changed
- Sum GPTBigCode's shared key/value gradient across tensor-parallel ranks by @Achyuthan-S in #8928
- [AffineIR] Storage maps of AutoEP experts under EDP and ZeRO by @jinyouzhi in #8941
- Add klxpu (KUNLUNXIN XPU) self-hosted CI for the v1 unit suite by @GuCarpenter in #8923
- ZeRO-3: copy offloaded master weights back to the device through pinned buffer by @pengdurice in #8840
- ci: pin torch<2.15 for MPS workflow until pytorch/pytorch#202529 is fixed by @delock in #8955
- [Hybrid Engine] Enable graph capture for HybridEngine on non-CUDA accelerators by @VenusTZZ in #8933
- Cap CPU accelerator memory at the cgroup limit of the process by @prabhkodes in #8930
- [VocabTP] vocab_parallel_ce support fused LM-head projection + loss for the HF forward by @jinyouzhi in #8945
- ZeRO-1/2: preserve gradients across request boundaries until step by @stas00 in #8949
- Update version.txt post-v0.19.9 release by @loadams in #8999
- Add an opt-in deterministic execution policy by @ziming-zh in #8917
- Keep float32 intermediates in the clamped SwiGLU eager fallback by @GuCarpenter in #8992
- ZeRO-3: build the partial-offload device string from the public accelerator API by @GuCarpenter in #8994
- [checkpoint] Execute affine transfer plans with bounded peer staging by @0z5a in #8809
- XPU: register pinned host memory via Level Zero ext_memmap by @delock in #8887
- [AutoTP] Support frozen AutoTP models and manage AutoTP mode per model by @jinyouzhi in #8851
- Fix torch-latest workflow unit failures by @sfc-gh-truwase in #8950
- Count each expert once in the fp32 gradient-clipping norm by @alanhuangyoo in #8478
- Support Transformers 5.20 cache query offsets in DeepSpeedStaticCache by @nathon-lee in #8852
- Fix bug in Adagrad optimizer initialization by @hwchen2017 in #8976
- Fix AutoEP storage-map transfer test for CPU tensor views by @jinyouzhi in #9012
- Clear stale offload scratch before restoring low-precision gradients by @adenzhou1350 in #8728
- Check that converting through an affine map writes every element once by @Achyuthan-S in #8975
- [Hybrid Engine] Require an explicit accelerator opt-in for graph capture by @VenusTZZ in #8980
- ZeRO-3: reuse the weight partition as the master copy when it already has the master dtype by @pengdurice in #8842
- AutoEP + AutoTP folding: allow CPU optimizer offload under ZeRO-3 by @0z5a in #8829
- Run the v1 multi-rank CPU suite on pull requests by @delock in #8944
- fix(rollout): pass a pre-built 4-D mask to the static graph decode forward by @anyadontfly in #9004
- AutoEP activation tests: skip the Triton grouped-GEMM path when Triton cannot run by @GuCarpenter in #8996
- Restore fp32 masters for ZeRO-3 and the ZeRO-1 BF16 optimizer by @sfc-gh-truwase in #8892
- AutoEP: read and write the transformers output-capture registry as a dict by @pengdurice in #9029
- CI: stop pip check from failing the AWS fallback on deepspeed-kernels by @pengdurice in #9020
- DeepNVMe: fix silent I/O failures on misaligned buffers by @xylian86 in #8985
- Keep fp32 CPU optimizer states in fp32 when loading a state_dict by @xylian86 in #8974
- fix test_multiple_request_boundaries_before_step by @stas00 in #9028
- ZeRO-3 NVMe offload: fix "param already assigned swap buffer" in the parameter prefetch by @xylian86 in #9014
- Fix routed-expert gradient clipping for AutoEP + AutoTP folding by @0z5a in #8823
- Treat an all-deselected selection as success in cpu-torch-latest-v1-multirank by @delock in #9032
- feat(rollout): non-greedy sampling on hybrid engine graph decode path by @anyadontfly in #8939
- [VocabTP] Add vocabulary-parallel generalized JSD with Triton kernels by @jinyouzhi in #8958
- ZeRO-1/2: one CPU-offload accumulator per group; zero idle gradients by @sfc-gh-truwase in #9018
- Persist ZeRO-3 HybridMegatron QKV layout changes by @LiRunGuo in #8392
- Fix memory leak in engine and zero optimizer by @hwchen2017 in #9034
- Code cleaning: remove commented out debug prints and code by @hwchen2017 in #9036
- Use C++20 for CUDA/ROCm op builders on torch 2.12+ by @VenusTZZ in #9041
- [NPU] Fix the fused LAMB step and inference int4/int8 dequantization on Ascend by @VenusTZZ in #9043
- Update Masahiro Tanaka’s affiliation in COMMITTERS.md by @tohtana in #9050
- ZeRO-2 CPU offload: accumulate in the compact partition between requests by @sfc-gh-truwase in #9044
- Synchronize ZeRO-3 rollout early stopping by @LiRunGuo in #8390
- Size HybridEngine rollout KV caches from local prefill shapes under AutoTP by @jinyouzhi in #9008
- Treat an empty subset selection as success in modal-torch-latest by @GuCarpenter in #9053
- Move test_klxpu_accelerator.py to tests/unit/v1/accelerator/ by @GuCarpenter in #8867
- klxpu: enable cuda graph capture by @GuCarpenter in #9042
- [checkpoint] Batch replica selection for affine transfer plans by @0z5a in #8936
- [AffineIR] Move shared affine index and box geometry into affine_geometry by @jinyouzhi in #9010
- Use the accelerator device in TestZero3SynchronizedEarlyStop by @dundysm in #9058
- Place restored step for fused and capturable optimizers on universal checkpoint load by @dundysm in #9056
- Add deepspeed.models.glm_moe_dsa: TileLang sparse attention for GLM-5.2 by @pengdurice in #8659
- Keep the parameters a model names for fp32 in fp32 under ZeRO-3 by @pengdurice in #8843
- Update version.txt before next release (bump minor version) by @loadams in #9072
New Contributors
- @prabhkodes made their first contribution in #8930
- @ziming-zh made their first contribution in #8917
- @anyadontfly made their first contribution in #9004
- @dundysm made their first contribution in #9058
Full Changelog: v0.19.9...v0.20.0