hpcaitech/ColossalAI v0.4.5

Version v0.4.5 Release Today!

on GitHub

What's Changed

Release

• [release] update version (#6094) by Hongxin Liu

Misc

• [misc] fit torch api upgradation and remove legecy import (#6093) by Hongxin Liu

Fp8

• [fp8] add fallback and make compile option configurable (#6092) by Hongxin Liu

Chore

• [chore] refactor by botbw

Ckpt

• [ckpt] add safetensors util by botbw

Pipeline

• [pipeline] hotfix backward for multiple outputs (#6090) by Hongxin Liu

Ring attention

• [Ring Attention] Improve comments (#6085) by Wenxuan Tan
• Merge pull request #6071 from wangbluo/ring_attention by Wang Binluo

Coati

• [Coati] Train DPO using PP (#6054) by Tong Li

Shardformer

• [shardformer] optimize seq parallelism (#6086) by Hongxin Liu
• [shardformer] fix linear 1d row and support uneven splits for fused qkv linear (#6084) by Hongxin Liu

Full Changelog: v0.4.5...v0.4.4