github microsoft/DeepSpeed v0.3.16
DeepSpeed v0.3.16

latest releases: v0.14.2, v0.14.1, v0.14.0...
3 years ago

v0.3.16 Release notes

  • Full precision (fp32) support for ZeRO Stage2 and Stage3 (#1004)
  • 1-bit LAMB optimizer (#970)
  • Refactor param_dict to config (#1008)
  • [ZeRO Infinity] Allow Init to take a dict for the deepspeed config (#983)
  • Bug fix for zero-1 w.r.t. reduce-scatter (#907)
  • Use amp autocast in ZeRO3 linear (#990)
  • Add find_unused_parameters option to DeepSpeedEngine (#945)
  • Relax dataset type check in deepspeed io (#1012)

Special thanks to our contributors: @stas00, @SeanNaren, @sdtblck, @wbuchwalter, @ghosthamlet, @zhujiangang,

Don't miss a new DeepSpeed release

NewReleases is sending notifications on new releases.