What's Changed
- ci: Update images for self-hosted ARM64 runner by @yongwww in #1128
- Fix pointer dtype bug in rope by @Edenzzzz in #1129
- feat: update and test create_ipc_buffer by @yyihuang in #1130
- misc: update runllm widget by @yzh119 in #1132
- misc: correct runllm widget (again) by @MasterJH5574 in #1133
- [Feature] Support PDL for batch Prefill and Decode by @Edenzzzz in #1117
- fix: negative zero by type trait --> binary value by @yyihuang in #1136
- fix: sync after create_workspace by @yyihuang in #1138
- refactor: use functools.cache instead of global dict for caching modules by @yzh119 in #1135
- [feat] add unified batch attention w/ correctness tests. by @happierpig in #1137
- Fix FA2 and FA3 multi-item scoring and cuda illegal memory access error by @arde171 in #1140
- feat: Add support for FLASHINFER_EXTRA_LDFLAGS environment variable by @jennifgcrl in #1144
- misc: remove sync between persistent runners and use packed_causal_kv_end for SM90Plan by @Edenzzzz in #1146
- [fix] fix precision errors when applying causal mask on Qwen-2.5 series models by @happierpig in #1148
- ci: Install mpi4py by @yongwww in #1149
- feat: add trtllm moe_allreduce_fusion by @yyihuang in #1108
- feat: add trtllm all-reduce fusion by @yyihuang in #1131
- Add more logging to TRTLLM-GEN debug trace (NFC) by @joker-eph in #1158
- feat: update non-fused moe by @yyihuang in #1161
- Add fp4 quantization swizzling tests by @wenscarl in #1157
- refactor: communication module by @yyihuang in #1162
- feat: add finalize_moe_allreduce from trtllm by @yyihuang in #1159
- feat: experimental support of green ctx by @yzh119 in #1163
- feat: Fused temperature online softmax kernel by @xslingcn in #1153
- MNNVL MoE All-to-All Support by @cyx-6 in #1134
- feat: nvshmem python bindings by @yzh119 in #1160
- Fix missing symbols in trtllm_utils.so by @tiran in #1168
- feat: logits processor fustion rule for temperature softmax by @xslingcn in #1170
- Expose fp4 blockscale swizzling kernel by @wenscarl in #1176
- add nvshmem sum_reduce for mnnvl allreduce by @Amir-19 in #1152
- bugfix: softmax NaN results caused by large -inf masks by @xslingcn in #1178
- [CI] Update is_last_build by @yongwww in #1183
- [feat] support block sparse attention w/ variable block sizes and head-wise sparse patterns by @happierpig in #1177
- bugfix: fix invalid blackwell fmha unittests by @yzh119 in #1181
- feat: support green ctx creation by a list of SM counts by @Conless in #1190
- fix: trtllm_comm module aot arch issues by @yyihuang in #1196
- bugfix: fix broken docs build by adding missing dependencies by @Conless in #1197
- chore: bump v0.2.7 by @zhyncs in #1199
New Contributors
- @jennifgcrl made their first contribution in #1144
- @tiran made their first contribution in #1168
- @Amir-19 made their first contribution in #1152
- @Conless made their first contribution in #1190
Full Changelog: v0.2.6.post1...v0.2.7