hpcaitech/ColossalAI v0.1.1 on GitHub

What's Changed

[MOE] changed parallelmode to dist process group by @1SAA in #460
[MOE] redirect moe_env from global_variables to core by @1SAA in #467
[zero] zero init ctx receives a dp process group by @ver217 in #471
[zero] ZeRO supports pipeline parallel by @ver217 in #477
add LinearGate for MOE in NaiveAMP context by @1SAA in #480
[zero] polish sharded param name by @feifeibear in #484
[zero] sharded optim support hybrid cpu adam by @ver217 in #486
[zero] polish sharded optimizer v2 by @ver217 in #490
[MOE] support PR-MOE by @1SAA in #488
[zero] sharded model manages ophooks individually by @ver217 in #492
[MOE] remove old MoE legacy by @1SAA in #493
[zero] sharded model support the reuse of fp16 shard by @ver217 in #495
[polish] polish singleton and global context by @feifeibear in #500
[memory] add model data tensor moving api by @feifeibear in #503
[memory] set cuda mem frac by @feifeibear in #506
[zero] use colo model data api in sharded optimv2 by @feifeibear in #511
[MOE] add MOEGPT model by @1SAA in #510
[zero] zero init ctx enable rm_torch_payload_on_the_fly by @ver217 in #512
[zero] show model data cuda memory usage after zero context init. by @feifeibear in #515
[log] polish disable_existing_loggers by @ver217 in #519
[zero] add model data tensor inline moving API by @feifeibear in #521
[cuda] modify the fused adam, support hybrid of fp16 and fp32 by @Gy-Lu in #497
[zero] refactor model data tracing by @feifeibear in #522
[zero] added hybrid adam, removed loss scale in adam by @Gy-Lu in #527

[MOE] add unitest for MOE experts layout, gradient handler and kernel by @1SAA in #469
[test] added rerun on exception for testing by @FrankLeeeee in #475
[zero] fix init device bug in zero init context unittest by @feifeibear in #516
[test] fixed rerun_on_exception and adapted test cases by @FrankLeeeee in #487

Full Changelog: v0.1.0...v0.1.1