• Adding MLA to MCore
• Enable FP8 for GroupedMLP
• MoE Parallel Folding
• Enhance MoE Architecture: Support MoE Layer Frequency Patterns and Configurable MoE FFN Hidden Size
• Multimodal: NVLM training and evaluation support in MCore
• Mamba Hybrid
  • Increase performance and reduce memory footprint of Triton language/compiler distributed caching
  • Add more unit testing and fix bugs