github NVIDIA/Megatron-LM core_v0.20.1
NVIDIA Megatron Core 0.20.1

one hour ago
  • Added Mixture-of-Kittens MoE backend (#8210)
  • TransformerEngine upgraded from 2.20 to 2.21
  • Various bug fixes
Changelog Details
  • beep boop 🤖: Bumping Megatron Core to v0.20.1 by @nvidia-megatron-lm-release-bot[bot] :: PR: #8714
  • cp: build: use PyPI nvidia-cudnn-frontend 1.29.1 in CI dev image (8303) into core_r0.20.0 by @ko3n1g :: PR: #8425
  • fix: backport MFSDP v2 EP checkpoint support to core_r0.20.0 by @cuichenx :: PR: #8465
  • cp: Refactor Schedule Node activation ownership (#8302) into core_r0.20.0 by @vthumbe1503 :: PR: #8911
  • build: update Transformer Engine to release_v2.21 tip on core_r0.20.0 by @balasaajay :: PR: #8792
  • Backport MoK, THD attention graphs, and DSv4 FLOPs to core_r0.20.0 by @cuichenx :: PR: #8929
  • cp: fix(pipeline): preserve input storage on core_r0.20.0 by @svcnemo-autobot :: PR: #9089
  • cp: Backport fused MLA KV RoPE MXFP8 quantization (#8144) to core_r0.20.0 by @malay-nagda :: PR: #9416
  • cp: fix(rl): backport FastAPI environment contract to core_r0.20.0 by @svcnemo-autobot :: PR: #9404
  • cp: Load the MLA query RoPE dims contiguously in the forward kernel (#8141) to core_r0.20.0 by @dingqingy-nv :: PR: #9487
  • cp: Accumulate the embedding weight gradient into main_grad (#8142) to core_r0.20.0 by @dingqingy-nv :: PR: #9486
  • build: fix NLTK CVEs on core_r0.20.0 by @balasaajay :: PR: #9492
  • build: upgrade Black to 26.10.1 on core_r0.20.0 by @balasaajay :: PR: #9496
  • docs: align core_r0.20.0 docs and latest pointer with 0.20.1 by @balasaajay :: PR: #9605
  • beep boop 🤖: Bumping Megatron Core to v0.20.1 [skip ci] by @github-actions[bot]
  • cp: build: use PyPI nvidia-cudnn-frontend 1.29.1 in CI dev image (8303) into core_r0.20.0 (#8425) by @ko3n1g
  • fix: backport MFSDP v2 EP checkpoint support to core_r0.20.0 (#8465) by @cuichenx
  • Refactor Schedule Node activation ownership (#8302) by @vthumbe1503
  • cp: Refactor Schedule Node activation ownership (#8302) into core_r0.20.0 (#8911) by @ko3n1g
  • build: update Transformer Engine to release_v2.21 tip on core_r0.20.0 (#8792) by @balasaajay
  • Backport MoK, THD attention graphs, and DSv4 FLOPs to core_r0.20.0 (#8929) by @cuichenx
  • cp: fix(pipeline): preserve input storage on core_r0.20.0 (#9089) by @svcnemo-autobot
  • cp: Backport fused MLA KV RoPE MXFP8 quantization (#8144) to core_r0.20.0 (#9416) by @malay-nagda
  • cp: fix(rl): backport FastAPI environment contract to core_r0.20.0 (#9404) by @svcnemo-autobot
  • cp: Load the MLA query RoPE dims contiguously in the forward kernel (#8141) to core_r0.20.0 (#9487) by @dingqingy-nv
  • cp: Accumulate the embedding weight gradient into main_grad (#8142) to core_r0.20.0 (#9486) by @dingqingy-nv
  • build: fix NLTK CVEs on core_r0.20.0 (#9492) by @balasaajay
  • build: upgrade Black to 26.10.1 on core_r0.20.0 (#9496) by @balasaajay
  • docs: align core_r0.20.0 docs and latest pointer with 0.20.1 (#9605) by @balasaajay

Don't miss a new Megatron-LM release

NewReleases is sending notifications on new releases.