hpcaitech/ColossalAI v0.1.12

Version v0.1.12 Release Today!

on GitHub

What's Changed

Zero

• [zero] add L2 gradient clipping for ZeRO (#2112) by HELSON

Gemini

• [gemini] get the param visited order during runtime (#2108) by Jiarui Fang
• [Gemini] NFC, polish search_chunk_configuration (#2107) by Jiarui Fang
• [Gemini] gemini use the runtime memory tracer (RMT) (#2099) by Jiarui Fang
• [Gemini] make RuntimeMemTracer work correctly (#2096) by Jiarui Fang
• [Gemini] remove eval in gemini unittests! (#2092) by Jiarui Fang
• [Gemini] remove GLOBAL_MODEL_DATA_TRACER (#2091) by Jiarui Fang
• [Gemini] remove GLOBAL_CUDA_MEM_INFO (#2090) by Jiarui Fang
• [Gemini] use MemStats in Runtime Memory tracer (#2088) by Jiarui Fang
• [Gemini] use MemStats to store the tracing data. Seperate it from Collector. (#2084) by Jiarui Fang
• [Gemini] remove static tracer (#2083) by Jiarui Fang
• [Gemini] ParamOpHook -> ColoParamOpHook (#2080) by Jiarui Fang
• [Gemini] polish runtime tracer tests (#2077) by Jiarui Fang
• [Gemini] rename hooks related to runtime mem tracer (#2076) by Jiarui Fang
• [Gemini] add albert in test models. (#2075) by Jiarui Fang
• [Gemini] rename ParamTracerWrapper -> RuntimeMemTracer (#2073) by Jiarui Fang
• [Gemini] remove not used MemtracerWrapper (#2072) by Jiarui Fang
• [Gemini] fix grad unreleased issue and param recovery issue (#2052) by Zihao

Hotfix

• [hotfix] fix a type in ColoInitContext (#2106) by Jiarui Fang
• [hotfix] update test for latest version (#2060) by YuliangLiu0306
• [hotfix] skip gpt tracing test (#2064) by YuliangLiu0306

Colotensor

• [ColoTensor] throw error when ColoInitContext meets meta parameter. (#2105) by Jiarui Fang

Autoparallel

• [autoparallel] support linear function bias addition (#2104) by YuliangLiu0306
• [autoparallel] support addbmm computation (#2102) by YuliangLiu0306
• [autoparallel] add sum handler (#2101) by YuliangLiu0306
• [autoparallel] add bias addtion function class (#2098) by YuliangLiu0306
• [autoparallel] complete gpt related module search (#2097) by YuliangLiu0306
• [autoparallel]add embedding handler (#2089) by YuliangLiu0306
• [autoparallel] add tensor constructor handler (#2082) by YuliangLiu0306
• [autoparallel] add non_split linear strategy (#2078) by YuliangLiu0306
• [autoparallel] Add F.conv metainfo (#2069) by Boyuan Yao
• [autoparallel] complete gpt block searching (#2065) by YuliangLiu0306
• [autoparallel] add binary elementwise metainfo for auto parallel (#2058) by Boyuan Yao
• [autoparallel] fix forward memory calculation (#2062) by Boyuan Yao
• [autoparallel] adapt solver with self attention (#2037) by YuliangLiu0306

Version

• [version] 0.1.11rc5 -> 0.1.12 (#2103) by Jiarui Fang

Pipeline middleware

• [Pipeline Middleware] fix data race in Pipeline Scheduler for DAG (#2087) by Ziyue Jiang
• [Pipeline Middleware] Adapt scheduler for Topo (#2066) by Ziyue Jiang

Fx

• [fx] An experimental version of ColoTracer.' (#2002) by Super Daniel

Example

• [example] update GPT README (#2095) by ZijianYY

Device

• [device] update flatten device mesh usage (#2079) by YuliangLiu0306

Test

• [test] bert test in non-distributed way (#2074) by Jiarui Fang

Pipeline

• [Pipeline] Add Topo Class (#2059) by Ziyue Jiang

Examples

• [examples] update autoparallel demo (#2061) by YuliangLiu0306

Full Changelog: v0.1.12...v0.1.11rc5