What's Changed
- MiCS implementation by @zarzen in #2964
- Fix formatting by @mrwyattii in #3343
- [ROCm] Hipify cooperative_groups headers by @rraminen in #3323
- Diffusers 0.15.0 bug fix by @molly-smith in #3345
- Print default values for DeepSpeed --help by @mrwyattii in #3347
- add bf16 cuda kernel support by @dc3671 in #3092
- README.md: Update MosaicML docs link by @kobindra in #3344
- hybrid_engine: check tuple size when fusing lora params by @adammoody in #3311
- fix mpich launcher issue in multi-node by @sywangyi in #3078
- Update DS-Chat issue template by @mrwyattii in #3368
- add deepspeed chat blog links, add tags by @conglongli in #3369
- Fix redundant shared_params in zero_to_fp32.py by @ShijieZZZZ in #3149
- fixing default communication_data_type for bfloat16_enabled and docs by @clumsy in #3370
- Auto TP Tutorial with T5 Example by @molly-smith in #2962
- stage_1_and_2.py: do gradient scale only for fp16 by @guoyejun in #3166
- Fix memory leak in zero2 contiguous gradients by @hablb in #3306
- remove megatron-lm, no longer pip installable by @jeffra in #3389
- Fix pipeline module evaluation when contiguous activation checkpoin… by @hablb in #3005
- doc updates by @goodship1 in #3415
- Save tensors in context of memory_efficient_linear by @tohtana in #3413
- Add HE support for the rest of model containers by @RezaYazdaniAminabadi in #3191
- Update PyTorch Lightning/DeepSpeed examples links by @loadams in #3424
- Fix
PipelineEngine.eval_batchresult by @nrailgun in #3316 - OPT Activation Function Hotfix by @cmikeh2 in #3400
- Add ZeRO 1 support to PP for BF16. by @jomayeri in #3399
- [zero_to_fp32] fix shared param recovery by @stas00 in #3407
- Adagrad support in ZeRO by @jomayeri in #3401
- Update 2020-09-09-sparse-attention.md by @goodship1 in #3432
New Contributors
- @dc3671 made their first contribution in #3092
- @kobindra made their first contribution in #3344
- @hablb made their first contribution in #3306
- @nrailgun made their first contribution in #3316
Full Changelog: v0.9.1...v0.9.2