flashinfer-ai/flashinfer v0.7.4rc1

Release v0.7.4rc1

on GitHub

What's Changed

• feat(prims-ts): broaden attention support and runtime split-KV by @PerkzZheng in #6925
• perf(cake_dsa_indexer): longest-segment-first unit tickets for multi-segment calls (sm_100a / sm_103a) by @yyihuang in #7953
• perf(cake_fmha): one-stage FP8 paged context programs for small per-request query counts by @yyihuang in #7954
• perf(cake_kda): one KCP workspace shared by prepared packs, with grad by @yyihuang in #7955
• ci: classify compatible public API input widening by @kangbintNV in #7787
• perf(cake_gqa): faster backward with per-thread TMEM release with grad by @yyihuang in #7958
• perf(cake_dense_projection_gemm): sm_107a rules for the thin and 512 / 1024 BF16 projection rows, and a spill-free 6144 -> 576 forward program by @yyihuang in #7960
• perf(cake_warp_decode): FC1/FC2 launch choices on the h3072_e256_i384_k8 decode rows on SM100/SM103 by @yyihuang in #7968
• fix(prims-ts): reuse attention kernels across Q/KV lengths by @PerkzZheng in #7106
• tests(cake_gdn): fix the GDN/KDA plan-parity test after the lazy KDA table upload by @yyihuang in #7970
• perf(cake_dsa_indexer): fuse the first two radix-select passes of the sampled sm_103a pair scan program by @yyihuang in #7971
• perf(cake_tinygemm2_tcgen05): add unbiased routing windows on sm_100a, sm_103a and sm_107a by @yyihuang in #7973
• feat(cake_gdn_det): sm_107a (R200) libraries with grad by @yyihuang in #7974
• feat(cake_vsa_sm90): route Sage-FP8 grouped-query pair pin vf plans to the shared-transposer queue program by @yyihuang in #7959
• feat(cake_lm_head_loss): ragged candidate log-probabilities and detached-teacher KL by @yyihuang in #7956
• perf(cake_moe_grouped_gemm): probed die split of the tile queues for the sm_103a dgrad down projection by @yyihuang in #7975
• perf(cake_kda): next-state prefetch, per-warp checkpoint stores and a B200 bin-mix build for BF16-input KDA prefill by @yyihuang in #7978
• perf(cake_kimi_k3_latent_moe): faster Kimi-K3 LatentMoE decode tails and prefill by @yyihuang in #7979
• perf(prims-ts): schedule D576 sparse MLA head counts between qualified ones by @PerkzZheng in #7984
• perf(cake_causal_conv1d): one halo exchange for a layer's q/k/v KCP convolutions by @yyihuang in #7985
• perf(cake_mhc): late-claim column-slab backward on SM100 so backward_finish runs under a full-SM hold by @yyihuang in #7988
• [Experimental] Qwen 3.8 Flash-Next block: the indexer's extend / prefill entries (chunked prefill through the serving caches) and the cuDNN Frontend 4c8e24355 re-vendor by @RomanAnders90 in #7983
• perf(cake_minimax_h3_varlen_attention): B200 two-stream NVFP4 two-launch schedules by @yyihuang in #7991
• perf(cake_deepgemm_mega_mhc): speed up sm_107a NVFP4 and FP8 group-128 raw-scale programs by @yyihuang in #7994
• perf(cake_kimi_k3_fp8_projection): SM103a resident fused decode for TP8 f_b M 257..512 by @yyihuang in #7992
• perf(cake_warp_decode): pre-wait-claims K512 FC2 sibling on 20 SM100 decode rows by @yyihuang in #7995
• perf(cake_sampling): lean-scan twin of the (4, 16) resident for V = 32768, top-k 769-1024 on GB300 and B200 by @yyihuang in #7989
• perf(cake_moe_grouped_gemm): probed die split of the tile queues for the sm_100a dgrad down projection by @yyihuang in #7998
• feat(benchmarks): allow prims-ts attention backends on SM107 by @lunarz-dev in #7997
• fix(cake_kda): record a shared KCP workspace's storages on a stream switch, with grad by @yyihuang in #8008
• perf(cake_dsa_indexer): fuse the first two radix-select passes of the sampled sm_100a scan program by @yyihuang in #8009
• perf(cake_fused_qk_rope_fp8_append): persistent stage for GB300 prefill of few long requests by @yyihuang in #8012
• perf(cake_kimi_k3_fused_router): faster one-token routing on B200 by @yyihuang in #8016
• perf(cake_tinygemm2_tcgen05): add 4 unbiased routing windows on sm_100a (B200) by @yyihuang in #8018
• perf(cake_gated_act_mxfp8): band-path program for the SM107 forward both entry below 2^20 by @yyihuang in #8023
• refactor(cake_gdn): deterministic KCP names for the context-parallel GDN autograd package by @yyihuang in #8022
• feat(cake_gdn): Add det to public entry-point names with grad by @yyihuang in #8027
• docs: fix v0.7.3 API reference gaps by @kangbintNV in #7993
• refactor(cake_mhc): deterministic names for the mHC autograd backend by @yyihuang in #8028
• Add MSA section to CODEOWNERS file by @aleozlx in #8039
• perf(cake_deepgemm): SM120 paged MQA-logits: route more shapes to tuned programs on RTX 5090 and RTX PRO 6000 by @yyihuang in #8029
• perf(cake_kimi_k3_fused_router): faster one-token routing on GB300 by @yyihuang in #8031
• ci: fail communication test wrappers on missing results by @righthandabacus in #8026
• perf(quantization): launch MXFP8 consumers earlier with PDL by @wolfcomos in #5982
• perf: hoist NVFP4 global reciprocal before the zero-scale check by @zianglih in #7976
• [Experimental] Qwen 3.5 block: the Qwen3.8-27B dense attention layer as a validated member and the packed K|V pool binding (kv_layout="UNIFIED") by @RomanAnders90 in #8036
• fix(msa_ops sm90): make the single-token CuTe prefill compile and run correctly on the cu12 CuTe DSL by @rmhaskarnvidia in #8037
• perf(topk): cache multi-CTA radix row_states per stream instead of zeroing per call by @aleozlx in #7964
• [SDSA 3/6][Indexer] Add paged MQA logits for a one-head indexer (BF16 keys) by @netanel-haber in #6422
• perf(cake_ulysses): faster world-8 bf16 direct scatter, two-unit zone copied in one round by @yyihuang in #8044
• perf(cake_minimax_h3): longest-first drain order for two-class overlapped SM120 NVFP4 varlen attention plans by @yyihuang in #8040
• refactor(cake_moe_grouped_det_gemm): deterministic names for the MoE grouped GEMM auto grad package by @yyihuang in #8043
• perf(cake_deepgemm_mega_mhc): speed up the sm_107a FP8 group-128 raw-scale programs by @yyihuang in #8047
• perf(cake_warp_decode): launch choices for two SM100 decode rows by @yyihuang in #8049
• perf(gemm): autotune PDL variants for mm_bf16 with pdl=None by @arpera in #8032
• test: fix staged MoE FFI signature checks by @righthandabacus in #8025
• refactor(cake_dense_projection_gemm): deterministic names for the dense projection GEMM package by @yyihuang in #8046
• [Experimental] Qwen 3.8 block: the extend / prefill entries take any row count (cuDNN Frontend 6b21be0e0 re-vendor) by @RomanAnders90 in #8058
• fix(grouped_mm): pass explicit offset boundaries for one expert on cuDNN >= 9.28 by @chengmingz-nv in #6853
• refactor(cake_rmsnorm): deterministic names for the RMSNorm and SiLU-gated RMSNorm auto grad packages by @yyihuang in #8045
• perf(cake_trtllm_moe_allreduce_union): one-token bf16 launches flush the peer publish with a fence warp on SM100/SM103 by @yyihuang in #8062
• Revert "perf(cake_minimax_h3): longest-first drain order for two-class overlapped SM120 NVFP4 varlen attention plans (#8040)" by @yyihuang in #8075
• perf(cake_moe_finalize_allreduce_fusion): issue the first token's route indices ahead of the control words by @yyihuang in #8073
• refactor(cake_dsa_indexer_det): deterministic names for the DSA indexer top-k package by @yyihuang in #8061
• perf(cake_deepgemm_mega_mhc): fast coefficient chain for two sm_107a tail programs by @yyihuang in #8084
• fix-a-cublas-kernel-issue by @yanqinz2 in #8087
• feat(cake_mega_moe): SM90 BF16 opt-in compressed expert weights (WCX) for decode rounds by @yyihuang in #8089
• refactor(cake_kda): give single-rank and context-parallel KDA training deterministic names by @yyihuang in #8054
• perf(cake_muse_spark_norm, cake_tp_moe_finalize_rmsnorm): retune the gated residual RMSNorm routes on GB300, the next-RMSNorm routes on B200 and R200, and the MoE finalize token-owner windows on B200, GB300 and R200 by @yyihuang in #8094
• feat(cake_lm_head_det_loss): det names for the chunked LM-head + loss backend by @yyihuang in #8059
• refactor(cake_causal_conv1d): deterministic KCP names for the context-parallel causal conv1d entry points by @yyihuang in #8055
• feat(cake_mhc): opt-in non-deterministic mHC autograd backend by @yyihuang in #8092
• fix(cake_mega_moe_v3): hang of the E384 T=1024 rank programs under GPU preemption by @yyihuang in #8095
• perf(cake_kimi_k3_latent_moe): faster Kimi-K3 LatentMoE TP1 decode tail on GB300 by @yyihuang in #8098
• feat(cake_rmsnorm_silu_gate): opt-in non-deterministic auto grad backend by @yyihuang in #8097
• feat(moe_ep): select Rubin MegaMoE variants with autotune v2 by @akaashrp in #7911
• feat(cake_moe_grouped_gemm): opt-in non-deterministic MoE grouped GEMM auto grad backend by @yyihuang in #8088
• perf(cake_deepgemm): SM120 paged FP8 MQA logits CTA-parallel scheduler routes for RTX 5090 and RTX PRO 6000 by @yyihuang in #8099
• perf(cake_fmha): multi-wave and short-KV one-stage FP8 paged context programs by @yyihuang in #8101
• perf(cake_trtllm_moe_allreduce_union): one-token bf16 launches at world size 2 on SM103 run the fence-warp program by @yyihuang in #8102
• fix(cake_mhc): no default mHC workspace allocation during CUDA graph capture by @yyihuang in #8100
• perf(cake_deepgemm_mega_mhc): fast coefficient chain for sm_107a UE8M0/NVFP4 tailpipes by @yyihuang in #8105
• perf(cake_causal_conv1d_det): add opt-in bwd_grid_mult backward split with grad by @yyihuang in #8104
• fix(cake_lm_head_det_loss): never leave a dX K-slice without a K step by @yyihuang in #8108
• perf(cake_tinygemm2_tcgen05): add sm_107a routing windows for 9-16-row unbiased GEMMs with 4097-16384 outputs by @yyihuang in #8110
• perf(cake_kimi_k3_fp8_projection): SM100a tail-N-split GEMM for TP8 o_proj by @yyihuang in #8111
• perf(cake_fused_qk_rope_append, cake_fused_qk_rope_fp8_append): add opt-in PDL launch on sm_100a/sm_103a by @yyihuang in #8112
• perf(cake_deepgemm_dense_mqa): add a parallel-schedule logits program for the partial-block qle128 kvsplit route on sm_100a up to K = 8192 by @yyihuang in #8113
• test(cake_lm_head_det_loss): compare the compacted logprob statistic in one CPU thread by @yyihuang in #8117
• perf(cake_kimi_k3_attn_res): retune dense routes on B200, B300 and GB300 by @yyihuang in #8120
• perf(cake_gdn_det_train): faster deterministic GDN chunked backward on long packs and HV = 3H by @yyihuang in #8121
• perf(cake_deepgemm_dense_mqa): extend the single-launch FP8 Q = 1 / Q = 128 and FP4 Q = 1 chains by one KV block by @yyihuang in #8124
• fix(cake_lm_head_det_loss): per-thread hidden-count state and a layout-keyed logprob backward binding by @yyihuang in #8126
• perf(cake_lm_head_det_loss): zero-fill the global sparse dW as FP32 words by @yyihuang in #8125
• perf(cake_ssd_combined): sm_107a seq_idx check program for single-sequence varlen calls by @yyihuang in #8132
• perf(cake_mega_moe_v3): faster TP/EP rank programs on sm_103a by @yyihuang in #8133
• [SM120] Paged FP8 MQA logits: route nine RTX 5090 h64 next_n 5/6 shapes to the native-atom programs by @yyihuang in #8136
• [Experimental] Qwen 3.5 GDN block: decode and MTP verify behind a backend registry, with an SM100 CuTe DSL backend by @YangXu1990uiuc in #8131
• perf(cake_moe_grouped_gemm): skip forward recompute under checkpoint early stop by @yyihuang in #8114
• perf(cake_mhc): run the non-deterministic backward's fn split beside the token-owner pass by @yyihuang in #8116
• fix(cake_tinygemm2_tcgen05): require CUDA 13.4 for the sm_107a programs by @yyihuang in #8123
• perf(cake_deepgemm_mega_mhc): faster sm_107a FP8 group-128 raw, UE8M0 det and BF16 det by @yyihuang in #8128
• feat(cake_causal_conv1d): opt-in non-deterministic causal conv1d auto grad backend by @yyihuang in #8134
• perf(cake_dense_projection_det_gemm): regenerate programs and use eight-warp input-gradient programs on sm_100a by @yyihuang in #8129
• perf(cake_dsv4): faster BF16 / FP8 sparse-MLA third follow-up on SM100 / SM103 by @yyihuang in #8138
• perf(cake_moe_finalize_allreduce_fusion): extend the small-token program to top_k 4 without a shared expert at world size 4 by @yyihuang in #8139
• perf(cake_fmha): FP8 paged context routes for short-KV and one-wave launches by @yyihuang in #8140
• perf(cake_deepgemm_dense_mqa): extend the FP4 single-launch scan classes by one KV block on sm_103a by @yyihuang in #8142
• perf(cake_dsa_indexer_det): retune host-side split policy and sm_107a l6 program choice by @yyihuang in #8145
• perf(cake_kimi_k3_attn_res): retune dense routes at M 384 and 512 on B200 and B300 by @yyihuang in #8143
• feat(cake_msa_icp_indexer): MSA-compatible index-context-parallel indexer for TP2/TP4, P128/P256, SM100/SM103/SM107 by @yyihuang in #8122
• feat(cake_mhc_det): with grad, add H=5120 and H=7168, faster H=4096 under side loads by @yyihuang in #8146
• perf(cake_mega_moe): WCX mid-round windows for three SM90 BF16 shapes by @yyihuang in #8147
• perf(moe_ep): skip masked slots in SM107 quantized combine by @akaashrp in #7910
• perf(cake_fused_qk_rope_append, cake_fused_qk_rope_fp8_append): faster enable_pdl=True decode launches on sm_100a/sm_103a by @yyihuang in #8149
• perf(cake_deepgemm_dense_mqa): serve small query counts one KV block past the fused Q <= 2048 bound on sm_103a by @yyihuang in #8150
• perf(cake_deepgemm_dense_mqa): serve fp4:le32:short on sm_100a up to K = 4096 with a bucket program of its own by @yyihuang in #8151
• perf(cake_mhc): two-size tail schedule for the non-deterministic column slab by @yyihuang in #8148
• perf(cake_deepgemm_dense_mqa): serve fp4:le32:short on sm_100a for Q <= 36 up to K = 32768 with its bucket program by @yyihuang in #8153
• fix(cake_lm_head_det_loss): make the binding cache and device constant cells safe across threads and streams by @yyihuang in #8152
• perf(cake_mega_moe_v3): faster TP rank programs on sm_103a by @yyihuang in #8154
• perf(cake_kimi_k3_attn_res): route M 384 K 4 on B300 to a packed-register-carry program and M 768 K 0 on B200 to the fast-math small-M program by @yyihuang in #8155
• perf(cake_deepgemm_mega_mhc): faster sm_107a FP8 group-128 UE8M0 and deterministic launches by @yyihuang in #8156
• perf(cake_alpha_moe): round-4 NVFP4, W8A8 and fused router kernels on sm_100a/sm_103a by @yyihuang in #8158
• perf(cake_kda_det_kcp): with grad, KCP fwd_o takes its (chunk, head) tiles from Cluster Launch Control by @yyihuang in #8159
• feat(cake_mhc): non-deterministic autograd at hidden 5120 and 7168 on SM100 / SM103 by @yyihuang in #8160
• perf(cake_gdn_det): with grad, overlap the HV = 3H split scan pass 2 with the fused tail on two streams by @yyihuang in #8161
• feat(cake_gdn): with grad, add non-deterministic Cake GDN chunked training backend by @yyihuang in #8157
• perf(cake_muse_spark_norm, cake_tp_moe_finalize_rmsnorm): retune the gated residual RMSNorm routes on GB300 and the MoE finalize token-owner windows on GB300, B200 and R200 by @yyihuang in #8162
• perf(cake_lm_head_det_loss): form the loss gradients only where a backward uses them by @yyihuang in #8163
• perf(cake_rmsnorm_silu_gate): standalone non-deterministic backend; backward claims row blocks late within one wave (#7548) by @yyihuang in #8164
• perf(cake_deepgemm_mega_mhc): sm_107a deterministic raw-scale launch at 1024 tokens on the tailpipe program by @yyihuang in #8167
• [SM120] Paged FP8 MQA logits: route 9 RTX 5090 and 7 RTX PRO 6000 shapes to faster existing routes by @yyihuang in #8166
• perf(cake_moe_grouped_gemm): walk the groups by rows, descending, in the unordered k512 wgrad by @yyihuang in #8168
• perf(cake_kimi_k3_attn_res): route M 512 K 4, M 1024 K 4 and M 512 K 5 on B300 to shipped persistent programs and M 1536 K 0 on B200 to the fast-math small-M program by @yyihuang in #8171
• perf(cake_causal_conv1d_det): exact products in the backward on the targets that measured faster by @yyihuang in #8169
• perf(cake_deepgemm_dense_mqa): serve fp8:q128:short and fp8:full:le32:short on sm_100a with a four-stage TMEM ring bucket program by @yyihuang in #8173
• feat(cake_gdn_kcp): with grad, add non-deterministic context-parallel Cake GDN chunked autograd backend by @yyihuang in #8172
• perf(cake_rmsnorm_silu_gate): non-deterministic long-input forward/backward routes and backward kernel update by @yyihuang in #8178
• docs(cake_mhc_det_train): side-stream token ranges at hidden 5120 / 7168 after #8175 by @yyihuang in #8185
• feat(cake_kda): opt-in lean local arena for the KCP backward (local_arena="lean", lean_passes=2) by @yyihuang in #8174
• perf(cake_kda): take the BF16 KDA prefill bin-mix build on sm_103a where its plan shortens the longest bin by @yyihuang in #8180
• perf(cake_mhc_det): H=5120/7168 backward_finish completes under full-SM side loads on sm_100a by @yyihuang in #8175
• perf(cake_msa_icp_indexer): faster sm_107a decode via early index-K prefetch and fused chain at page size 128 by @yyihuang in #8177
• perf(cake_gdn): with grad, re-plan sm_107a scan segment tables for the R200 SM count by @yyihuang in #8181
• perf(cake_causal_conv1d): faster dweight flush in the non-deterministic autograd backward by @yyihuang in #8176
• perf(cake_deepgemm_dense_mqa): serve fp8:qle2048:short on sm_103a for Q <= 136 up to K = 8448 with a four-stage TMEM ring bucket program by @yyihuang in #8183
• perf(cake_fmha): faster FP8 paged context for short-KV and one-wave launches by @yyihuang in #8184
• perf(cake_rmsnorm_det): H 5120, per-head H 256 and FP32-weight autograd RMSNorm (SM100 / SM103 / SM107) by @yyihuang in #8187
• perf(cake_mhc): with grad, two-size column slab schedule at hidden 5120 / 7168 on sm_100a by @yyihuang in #8189
• perf(cake_sampling): H100 L2-prefetch twins for top-k/top-p sampling at vocab 262144, batch 128 by @yyihuang in #8182
• feat(cake_dense_projection_gemm): non-deterministic split-K and stream-K autograd GEMMs by @yyihuang in #8190
• perf(cake_mega_moe_v3): faster TP4, TP8, TP4 EP4 and TP8 EP8 rank programs at 4096 tokens on sm_103a by @yyihuang in #8192
• perf(cake_gdn): with grad, split two-stream adjoint scan for sm_107a HV=3H; lighter host path by @yyihuang in #8194
• fix(cake_gdn): keep BF16 out of the SM107 Cake CP auto band and fail closed on long-sequence FP32-state rows by @yyihuang in #8196
• fix(cake_rmsnorm_det): with grad, dw reproducibility per SM count; compensated backward below the measured SM counts by @yyihuang in #8195
• perf(cake_lm_head_det_loss): opt-in kept logits for the log-probability backward by @yyihuang in #8197
• fix(cake_deepgemm_mega_mhc): one launch for hidden-4096 batches above 64 tokens/SM by @yyihuang in #8199
• cake_gdn_cp: regenerate the SM107 T-precompute program with checked host descriptor arithmetic by @yyihuang in #8200
• perf(cake_selective_state_update): the z-gated one-token ring at its measured batch on sm_103a by @yyihuang in #8203
• perf(cake_causal_conv1d_det_train): row-group backward program for 264 <= C < 512 on SM100 / SM103 by @yyihuang in #8205
• refactor(cake_nvfp4_svdquant): label registry records by kernel origin by @yyihuang in #8204
• perf(cake_kimi_k3_attn_res): route M 4096 to 8192 with K 6 to 8 on B200 and B300 to token-queue persistent programs by @yyihuang in #8207
• cake_gdn_cp: dedicated sm_107a ring state-fixup program (about 1.05x) by @yyihuang in #8206
• perf(cake_tinygemm2_wgmma): sm_90a wgmma programs for tinygemm_bf16, bit-identical to csrc/tinygemm2.cu by @yyihuang in #8201
• perf(cake_msa_icp_indexer): sm_107a fused decode list scorer prefetches index-K before the PDL wait by @yyihuang in #8210
• perf(cake_trtllm_mla_blackwell): faster quarter-V BF16 and GB300 152-SM CLC decode programs by @yyihuang in #8211
• perf(cake_gdn_kcp): with grad, lighter host path for the non-deterministic KCP package by @yyihuang in #8212
• fix(cake_ssd_combined): refuse a seq_idx triple over more than one row by @yyihuang in #8213
• chore(cake_lm_head_det_loss): re-render the package from the merged Cake revision by @yyihuang in #8214
• cake_gdn: route the remaining SM107 FP32-state head splits to the gate-streaming prefill body (up to 1.12x) by @yyihuang in #8216
• perf(cake_rmsnorm_silu_gate): non-deterministic mid-input forward on SM100/SM103 by @yyihuang in #8217
• perf(cake_gdn): overlap the two compute warpgroups in the cross-rank state walk by @yyihuang in #8215
• perf(cake_gdn_kcp): with grad, cut more eager host overhead in non-deterministic KCP by @yyihuang in #8219
• perf(cake_mega_moe_v3): faster TP4 / TP8 T32 and TP4 EP4 / TP8 EP8 rank programs on sm_103a by @yyihuang in #8220
• fix(cake_lm_head_det_loss): correct selective activation checkpointing of the det entries by @yyihuang in #8222
• perf(cake_deepgemm_mega_mhc): sm_107a raw launches at 129-384 and 833-1023 on tailpipe by @yyihuang in #8223
• feat(cake_kda): with grad, add non-deterministic Cake KDA chunked autograd backend by @yyihuang in #8191
• perf(cake_moe_grouped_gemm): finish the unordered k512 wgrad tail by whole rows and reduce it in lane pairs by @yyihuang in #8198
• perf(cake_fmha): faster B200 FP8 context split-KV launches with two splits by @yyihuang in #8202
• feat(cake_gdn): run unshipped SM107 DV-split prefill keys on their full-DV records (up to 3.01x) by @yyihuang in #8225
• perf(cake_kda): build every record of a rebound BF16 KDA prefill launch before the first kernel by @yyihuang in #8228
• perf(cake_dsa_indexer_det): sm_103a CTA-pair l6 scan programs walk key tiles in descending order by @yyihuang in #8224
• perf(cake_kda): with grad, checkpoint hand-off for the non-deterministic Cake KDA autograd backend by @yyihuang in #8209
• perf(cake_megamoe_topk_reduce): admit sm_107a (R200) with its own token tiers by @yyihuang in #8229
• perf(cake_comm): QkvNormRopeExchangeOverlap SM107 two- and four-rank plans publish one readiness word per destination by @yyihuang in #8231
• perf(cake_tinygemm2_wgmma): faster sm_90a packed and accumulator programs, wider routing windows by @yyihuang in #8232
• feat(cake_kda): with grad, add non-deterministic Cake KDA context-parallel (KCP) autograd backend by @yyihuang in #8218
• perf(cake_gdn): SM107 CP prefill T-precompute on named barrier 1 and a 3-CTA ring state fixup (up to 1.18x) by @yyihuang in #8234
• feat(cake_rmsnorm): opt-in non-deterministic weight gradient for the autograd RMSNorm by @yyihuang in #8208
• perf(cake_rmsnorm_silu_gate): non-deterministic bulk-prefetch long backward on SM100/SM103 by @yyihuang in #8235
• fix(cake_softmax): round the Blackwell softmax exponent once by @yyihuang in #8236
• perf(cake_selective_state_update): L2 read-ahead builds of the lean P-split decode programs at their measured shapes on sm_100a by @yyihuang in #8237
• perf(cake_dense_projection_gemm): head/tail stream-K router wgrad and R200 split-K routes by @yyihuang in #8238
• perf(cake_nvfp4_mla_decode): faster B200 long-row decodes and sm_100a two-split combines by @yyihuang in #8239
• perf(cake_fmha): aligned hd64 q1 grouped paged decode program for sm_100a / sm_103a by @yyihuang in #8241
• perf(cake_mhc): with grad, slab-fastest column-slab order in the nd backward on sm_100a / sm_103a by @yyihuang in #8240
• perf(cake_fused_qk_rope_append): per-call CTA cap for the BF16 8/1 prefill PDL builds on B200 and GB200 by @yyihuang in #8242
• perf(cake_mega_moe_v3): faster TP4 / TP8 T1024, TP4 T4096, TP8 T16 and TP8 EP8 rank programs on sm_103a by @yyihuang in #8244
• feat(cake_lm_head_loss): non-deterministic chunked LM-head + loss package by @yyihuang in #8193
• perf(cake_dsa_indexer_det): skip the ticket-tail clear on static calls; sm_107a split and wide program rules by @yyihuang in #8243
• perf(prims-ts): run sparse MLA on SM107 and trim the NVFP4 KV converter by @PerkzZheng in #8186
• perf(cake_dsa_indexer_det): sm_100a and sm_103a l6 scan programs walk key tiles in descending order by @yyihuang in #8246
• perf(cake_deepgemm): faster SM120 paged FP8 MQA logits routes for six RTX 5090 shapes by @yyihuang in #8245
• perf(cake_moe_grouped_det_gemm): skip forward recompute under checkpoint early stop by @yyihuang in #8247
• perf(cake_mega_moe): sm_107a (R200) build of the BF16 rank-major EP8 layer by @yyihuang in #8249
• feat(cake_mxfp8_megamoe_ep16): admit sm_107a (R200) for both backends and regenerate the CuTe-DSL backend by @yyihuang in #8248
• perf(cake_moe_grouped_gemm): run the split k512 wgrad tail whole on the least-loaded clusters on sm_107a by @yyihuang in #8258
• feat(moe_ep): support MXFP4 W4A4 on SM107 by @akaashrp in #7944
• perf(cake_kimi_k3_latent_moe): faster Kimi-K3 LatentMoE TP8 decode tail with two routed partials on GB300 by @yyihuang in #8263
• test(cake_lm_head_det_loss): device-independent test inputs and a first-use-free cache-miss measurement by @yyihuang in #8262
• perf(cake_mhc_det): with grad, group-row token owner and reduction above 14927 tokens on sm_100a / sm_103a by @yyihuang in #8256
• fix(cake_rmsnorm_det): with grad, improve large-row weight-gradient accuracy by @yyihuang in #8251
• perf(cake_kda): BF16 KDA prefill split-form mains read the caller's window indices, dropping the index-prep launch by @yyihuang in #8265
• perf(cake_msa): Blackwell uniform FP8 paged decode programs chosen by batch and KV length, plus a fused large-batch route by @yyihuang in #8264
• perf(cake_dense_projection_gemm): head/tail router weight gradient on sm_107a at hidden 5120 by @yyihuang in #8261
• fix: replace M_LOG2E with portable typed constants by @saltyminty in #8051
• perf(cake_ssd_combined): sm_107a batched-view routes for long single-sequence prefill and 4-chunk batched calls by @yyihuang in #8267
• perf(cake_gdn_det): with grad, overlap the fused backward tail with the adjoint scan by @yyihuang in #8271
• perf(cake_deepgemm_mega_mhc): sm_107a raw FP8 launches at 969-1023 tokens on a chain-packing program by @yyihuang in #8274
• perf(cake_rmsnorm_silu_gate): non-deterministic short-input forward on SM100/SM103 by @yyihuang in #8270
• perf(cake_causal_conv1d): faster non-deterministic autograd backward (row-group program, batched finish, fence-free ticket) by @yyihuang in #8257
• perf(prims-ts): unroll runtime-split MLA reduction in dispatched windows by @PerkzZheng in #8103
• feat(cake_kda_det): with grad, raw forget-gate inputs on the single-rank route (#7816) by @yyihuang in #8276
• feat(cake_gdn): in-kernel Q/K L2 norm for the deterministic KCP GDN autograd package (#7817) by @yyihuang in #8252
• feat(prim_ts): Add H512 FP8-Q/NVFP4-KV decode support by @xuanzic in #5720
• feat(dsa_indexer): experimental SM120 backend for FP8 / MXFP4 prefill and decode by @lucifer1004 in #7085
• perf(cake_moe_grouped_gemm): run the split k512 wgrad tail whole on the least-loaded clusters on sm_100a and sm_103a by @yyihuang in #8269
• perf(cake_dense_projection_gemm): 192-wide row-major FP32 weight-gradient program for 4096 -> 1024 on 212-SM sm_107a by @yyihuang in #8275
• feat(moe): run the FP4 routed MoE expert body from caller-populated routing metadata by @wookjeHan in #8050
• perf(cake_mxfp8_megamoe_ep16): 192-CTA grid for the sm_107a CUDA backend by @yyihuang in #8278
• perf(cake_gdn): small_one as the default segment plan rule for the context-parallel GDN autograd package by @yyihuang in #8279
• perf(cake_softmax): one register-tile max in the widest Blackwell row-caching kernels by @yyihuang in #8280
• perf(cake_warp_decode): tile-N8 FC2 programs for sm_100a decode rows by @yyihuang in #8285
• perf(cake_alpha_moe): round-5 NVFP4 one-token up on sm_103a and W8A8 deep-ring pair kernel on sm_100a/sm_103a by @yyihuang in #8283
• perf(cake_mhc_det): with grad, xrsg_p160 column-slab program at the sm_100a fit rows by @yyihuang in #8266
• perf(cake_msa_icp_indexer): cheaper decode candidate merge on sm_100a, sm_103a and sm_107a by @yyihuang in #8286
• perf(cake_deepgemm_dense_mqa): single-launch small-Q bands on sm_100a and the TMEM4 fp8:q128:short bucket on sm_103a by @yyihuang in #8290
• perf(cake_gdn): reuse recomputed normalized Q/K in the checkpointed backward of the deterministic KCP GDN package by @yyihuang in #8288
• fix(cake_rmsnorm_det): tighten two large-row weight-gradient accuracy edges by @yyihuang in #8293
• feat(cake_kda_det_kcp): with grad, raw forget-gate inputs on the KCP route (#7816) [stacked] by @yyihuang in #8255
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM100 16384-token row: finalize with every routed-slot load in flight by @yyihuang in #8289
• fix(cake_kda): with grad, re-copy the carried programs after the deterministic re-render (#8276) by @yyihuang in #8292
• perf(cake_rmsnorm_silu_gate): non-deterministic one-cluster short backward on SM100/SM103 by @yyihuang in #8301
• perf(cake_kimi_k3_attn_res): faster programs for more B200, B300 and GB300 shapes by @yyihuang in #8291
• perf(cake_tinygemm2_wgmma): sm_90a packed and accumulator ring fill reordered, interleaved packed tiles, re-measured routing by @yyihuang in #8306
• feat(cake_lm_head_det_loss): scalar and per-token temperature for chunked_lm_head_det_logprob by @yyihuang in #8254
• perf(cake_mhc): with grad, B200 PDL bands in the nd training route by @yyihuang in #8281
• test(cake_dense_projection_gemm): run the 192-wide row-major GPU test on sm_107a only by @yyihuang in #8313
• perf(cake_moe_finalize_allreduce_fusion): two-shot early programs for batches of up to 128 tokens at world sizes 4 and 8 by @yyihuang in #8321
• perf(cake_tinygemm2_wgmma): sm_90a routing windows extended between grid values by @yyihuang in #8324
• feat(cake_lm_head_det_loss): sm_107a (R200) record with every sm_103a program by @yyihuang in #8260
• perf(cake_dense_projection_gemm): router weight gradient tables for 208-SM and 212-SM sm_107a parts by @yyihuang in #8310
• perf(cake_mhc): with grad, GB300 column-slab PDL bands in the nd autograd route by @yyihuang in #8304
• perf(cake_mega_moe_v3): faster TP8 T4096 rank program on sm_103a by @yyihuang in #8327
• feat(cake_gdn_kcp): with grad, in-kernel Q/K L2 norm for the non-deterministic KCP package (#7817) by @yyihuang in #8316
• perf(cake_causal_conv1d): reuse the non-deterministic single-rank launch plan per call row; coalesced FP64 flush on sm_103a bias + FP32-gradient backward by @yyihuang in #8323
• perf(cake_rmsnorm_silu_gate): non-deterministic mid- and long-input forwards on SM107 by @yyihuang in #8332
• perf(cake_dense_projection_gemm): leaner deterministic=False convenience path for the router GEMM by @yyihuang in #8309
• perf(cake_fmha): aligned hd256 q1 grouped paged decode programs for sm_100a / sm_103a by @yyihuang in #8337
• perf(cake_mhc): GB200 PDL bands for the non-deterministic mHC autograd kernels by @yyihuang in #8317
• perf(cake_mok_bf16): re-render the sm_107a MXFP8 autograd kernels (#7821) by @yyihuang in #8302
• perf(cake_gdn): run ragged SM107 FP32-state prefill batches bounded by their longest sequence on the DV-split body (up to 1.17x) by @yyihuang in #8338
• perf(cake_kimi_k3_latent_moe): faster Kimi-K3 LatentMoE TP8 and TP1 multi-partial decode tails on GB200 and GB300 by @yyihuang in #8340
• perf(cake_dsa_indexer_det): use unpaired l6 scan for sampling-off calls <= 16384 queries by @yyihuang in #8339
• perf(cake_lm_head_loss): skip the zero-gradient rows of the log-probability backward by @yyihuang in #8268
• perf(cake_deepgemm_mega_mhc): chain-packing programs for sm_107a FP8 launches on 208-SM devices by @yyihuang in #8341
• perf(cake_gdn_det): with grad, overlap the fused backward tail with multi-round split scans on B200 by @yyihuang in #8342
• perf(cake_trtllm_mla_blackwell): cap FP8 page-64 reducer residency with a dynamic shared memory reservation by @yyihuang in #8343
• test(cake_mhc): with grad, nd tests follow the deterministic group-row forms by @yyihuang in #8333
• perf(cake_mhc_det_train): per-SM capped ticket claim of the mapping pass at the uneven sm_100a rows by @yyihuang in #8335
• perf(cake_rmsnorm_det): faster H 256 dw reduce and dc main, register-compensated H 1536 backward by @yyihuang in #8319
• perf(cake_dense_projection_gemm): deterministic K1 weight gradient at measured slow or less accurate token counts on sm_100a / sm_103a by @yyihuang in #8326
• perf(cake_mok_bf16): quantize both MXFP8 weight layouts from one read (#7821) by @yyihuang in #8346
• Fix test_joint_config_validation_uses_gathered_backends when no GPUs are visible by @akshay919191 in #7719
• fix(attention): make FA2 paged-prefill stride dispatch portable by @saltyminty in #8048
• perf(cake_kimi_k3_attn_res): token-queue and register-carry programs for more GB300 and B200 shapes by @yyihuang in #8351
• chore: add Quantization section to CODEOWNERS by @kahyunnam in #7946
• perf(quantization): add register-resident CUDA backend by @NDNM1408 in #5546
• perf(cake_kda): BF16 KDA prefill sm_103a scalar-beta map fold keeps the gathered epilogue spill-free by @yyihuang in #8352
• perf(cake_alpha_moe): NVFP4 one-token up merges its K splits in a cluster on sm_100a by @yyihuang in #8353
• perf(cake_kda_det): short block-table segments under side-stream contention, dqkg smem fence (#7548) by @yyihuang in #8328
• fix(cake_dsa_indexer_det): ack the verdict word before the next unit stores it by @yyihuang in #8355
• fix(cake_kda): with grad, follow the deterministic block-table and fence re-render in the non-deterministic package [stacked on #8328] by @yyihuang in #8347
• feat(moe): support ClampedRelu2 in the CuTe-DSL fused MoE by @danisereb in #7286
• perf(cake_mxfp8_megamoe_ep16): 104-cluster schedule for the sm_107a CuTe-DSL backend by @yyihuang in #8330
• perf(cake_msa_icp_indexer): faster list publisher and early merge trigger on sm_100a, sm_103a and sm_107a by @yyihuang in #8358
• perf(cake_gdn_kcp): with grad, reuse recomputed normalized Q/K in the checkpointed backward of the non-deterministic KCP package by @yyihuang in #8349
• fix(comm): wait on the cluster barrier before the MNNVL RMSNorm DSMEM exchange by @MatthewBonanni in #7782
• fix(cake_dense_projection_det_gemm): take the cluster rendezvous before the 2-CTA tensor-memory allocation, and add sm_107a row rules by @yyihuang in #8303
• perf(cake_mxfp8_megamoe_ep16): 104-cluster schedule for the sm_107a CUDA backend by @yyihuang in #8318
• test(cake_moe_grouped_gemm): keep the split-tail shape search within the kernel's 32-bit output span by @yyihuang in #8360
• perf(prims-ts): better sliding-window masks for Prims-ts context attention by @IwakuraRein in #7773
• perf(cake_mega_moe_v3): faster TP4 EP4 and TP8 EP8 T128 rank programs on sm_103a by @yyihuang in #8362
• fix(mamba): preserve FP16 subnormals in software stochastic rounding by @adenzhou1350 in #6114
• perf(cake_warp_decode): select tile-N8 FC2 programs per SM103 decode row by @yyihuang in #8370
• perf(cake_alpha_moe): NVFP4 token-tile down splits each tile three ways on sm_100a at 6K-11K tokens by @yyihuang in #8371
• perf(cake_gdn_det_kcp): run the summaries' offset and transfer walks concurrently in the deterministic KCP GDN autograd package by @yyihuang in #8348
• perf(cake_rmsnorm_det): with grad, faster H5120 backward on sm_107a with TwoSum weight-gradient reduction by @yyihuang in #8350
• perf(cake_kda): with grad, checkpoint hand-off for the non-deterministic Cake KDA context-parallel (KCP) autograd backend by @yyihuang in #8366
• perf(cake_dense_projection_gemm): run the deterministic program at the sm_107a token counts its new plans win by @yyihuang in #8363
• perf(cake_tinygemm2_wgmma): sm_90a routing re-selected for multi-wave accumulator grids and gap shapes by @yyihuang in #8376
• perf(cake_tinygemm2_wgmma): paced ring fill for the sm_90a eight-stage cluster programs by @yyihuang in #8377
• perf(cake_dense_projection_gemm): skip the per-call verdict check of router rows without a verdict by @yyihuang in #8373
• perf(cake_gdn_kcp): pair the summaries' walks by default on sm_100a / sm_103a in the non-deterministic KCP package by @yyihuang in #8364
• perf(cake_dsv4): SM100 BF16 H64 compressed Q8 decode stores partial O through a TMA map by @yyihuang in #8381
• fix(gemm): enable the TinyGEMM mm_bf16 backend on SM107 by @arpera in #8227
• refactor(gemm): drop the cuBLASLt fallback from the mm_bf16 cute-dsl backend by @arpera in #8230
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM100 64-token row: quantize and route in one launch by @yyihuang in #8383
• perf(cake_mhc_det): with grad, dual-layout fn split in one launch for the activation-checkpoint recompute by @yyihuang in #8312
• feat(cake_fp8_rowwise_gemm_allreduce): SM100 fused FP8 GEMM + all-reduce for TP2/4/8 by @yyihuang in #8379
• chore(codeowners): add tinygemm2 files to GEMM owners by @jimmyzho in #8394
• fix(tinygemm2): compute the output index in 64 bits by @arpera in #8334
• docs(cake_gdn): plain split views of packed Q/K/V cost a concatenation in the deterministic KCP backward by @yyihuang in #8385
• perf(cake_gdn_kcp): with grad, keep the row_cost_lpt plan for long one-chain fragments of the non-deterministic KCP plan on R200 by @yyihuang in #8357
• perf(cake_gdn): route SM107 GDN decode to faster bitwise-identical kernel bodies (up to 1.18x) by @yyihuang in #8382
• feat(cake_lm_head_loss): sm_107a (R200) record with every sm_103a stage by @yyihuang in #8282
• fix(cake_fmha): plan small-M decode splits on the launch stream's SM partition by @yyihuang in #8387
• perf(cake_mega_moe_v3): faster TP4 EP4 and TP8 EP8 T512 and T1024 rank programs on sm_103a by @yyihuang in #8388
• feat(batch_invariant): batch-invariant mode with mm_bf16(batch_invariant=True) by @bkryu in #8361
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM100 single-token row with evict-first weight-stream loads by @yyihuang in #8392
• Update CODEOWNERS by @aleozlx in #8411
• feat(prims-ts): QK-MXFP8 context FMHA by @harrisonzhy in #7509
• perf(cake_softmax): shorter max-to-exponent chain in the Blackwell 4-tile register softmax kernels by @yyihuang in #8390
• fix(cake_fmha): sync the warp after the NVFP4 context JIT member's page_empty wait by @yyihuang in #8408
• test(cake_fused_moe): Kimi-K3 NVFP4 SiTU 64-token fused quant+route bitwise case for hot experts by @yyihuang in #8409
• fix(cake_fmha): BF16 decode programs read no V page past a request's length by @yyihuang in #8407
• fix(cake_fmha): BF16 decode page tables, retained-KV sink grid, the exact B128 retained-KV member and a correction-warp race by @yyihuang in #8406
• [SDSA] Add code owners for the SDSA indexer by @netanel-haber in #8380
• docs(cake_gdn_kcp): plain split views of packed Q/K/V cost a concatenation in the non-deterministic KCP backward by @yyihuang in #8393
• perf(cake_tp_moe_finalize_rmsnorm): add cluster-pair token-owner programs for B200 TP8 and GB300 TP4 by @yyihuang in #8413
• chore: add misc-op CODEOWNERS and collapse op rules to directory-level globs by @kahyunnam in #8396
• fix(cake_deepgemm_dense_mqa): split paged FP8 MQA logits past 2^31 elements into request ranges by @yzh119 in #8315
• fix(cake_softmax): admit sm_107a with PDL, and read the rowwise temperature after the PDL wait by @yzh119 in #8369
• ci(release): pass release notes by file and cap them at the GitHub body limit by @aleozlx in #7972
• perf(msa_ops sm90): bound the indexer's work by the real sequence length under CUDA graphs by @rmhaskarnvidia in #8056
• feat(cake_deepgemm_mega_gate): token mask, image bias, per-call PDL, decode templates and SM107a as a vendorable package by @yzh119 in #8320
• fix(cake_deepgemm_dense_mqa): split 32-head FP8 MQA logits past 2^32 elements into row ranges by @yzh119 in #8308
• fix(msa_ops): serve SM90 sparse-decode coordinates whose planned route was never delivered (#7222) by @aleozlx in #8014
• fix(attention): update TRTLLM-Gen FMHA cubins for grouped NVFP4 by @PerkzZheng in #8115
• fix(cake): allocate pair tensor memory after the cluster barrier in four more families by @yzh119 in #8384
• perf(cake_dsv4): route FP8 64-head sparse-MLA decode rows to the M64 body once two CTAs per token exceed the SMs by @yzh119 in #8311
• fix(cake_fp4_allreduce): canonical wire words for pre-quantised NVFP4 input, call-index wrap, async-proxy fence after the PDL wait by @yzh119 in #8325
• perf(cake_tinygemm2_wgmma): sm_90a small-batch shapes routed to the paced cluster programs by @yyihuang in #8417
• perf(cake_dsv4): programmatic dependent launch for the FP8 persistent and M64 sparse-MLA programs (stacked on #8311) by @yzh119 in #8314
• perf(cake_dense_projection_det_gemm): swapped small-token forward programs with per-call PDL by @yzh119 in #8372
• feat(comm): add SageAttention2 low-precision QKV exchange to UlyssesCommunicator by @DwenGu in #4924
• perf(cake_deepgemm_dense_mqa): raw-store programs for clean_logits=False on the 32-head FP8 routes by @yzh119 in #8329
• fix(cake_fmha): give the FP8/FP16 context JITs' final row statistics their own barrier by @yyihuang in #8424
• perf(cake_kimi_k3_attn_res): token-queue and register-carry programs for more B200, B300 and GB300 shapes by @yyihuang in #8428
• perf(cake_deepgemm_mega_mhc): SM-count-keyed sm_107a routes to existing tailpipe programs on R200 by @yyihuang in #8425
• perf(cake_gdn_kcp): fold the output program into the forward walks of the non-deterministic KCP package by @yyihuang in #8421
• perf(cake_lm_head_loss): counted TMA dW cast for a short last chunk and a shorter skip host path by @yyihuang in #8426
• perf(cake_fmha): aligned MTP q32 grouped paged decode program for sm_100a / sm_103a by @yyihuang in #8415
• feat(cake_fp4_allreduce): four-rank fused all-reduce + residual + RMSNorm; any row count at four ranks by @yzh119 in #8395
• perf(cake_dense_projection_gemm): launch the deterministic router program through this package's binding by @yyihuang in #8430
• perf(cake_dense_projection_det_gemm): add GB200 / GB300 T-exact row rules for the thin input gradient, the N 512 forwards and the 6144 x 2048 rows by @yyihuang in #8441
• feat(cake_dsv4): SM107 runs the FP8 decode route of backend='cake'; the low-head prefill program takes PDL by @yzh119 in #8375
• perf(cake_fmha): refit sm_100a FP8 paged-context route selection by @yyihuang in #8446
• fix(cake_fmha): cache the small-M partition SM count per CUDA stream ID by @yyihuang in #8420
• fix(cake_fmha): keep the FP8 decode programs' page-32 table slots in the stage by @yyihuang in #8445
• perf(cake_mhc): with grad, slab-order column slab on 212-SM R200 boards in the non-deterministic route by @yyihuang in #8410
• perf(cake_fmha): opt-in length-aware BF16 decode schedule by @yyihuang in #8447
• perf(cake_gdn_det): elected ready-count publish for split-scan overlap; enable on GB300 by @yyihuang in #8448
• perf(cake_dsv4): 256-bit O stores in fp8_lowhead_one_partition and a column-split O drain in bf16_swa128_single_cta on sm_100a by @yyihuang in #8452
• fix: update FMHA cubins for grouped NVFP4 decode by @saltyminty in #8422
• perf(cake_gdn_cp): native BF16 GDN CP prefill on sm_107a (up to 1.35x) by @yyihuang in #8457
• feat(cake_lm_head_loss): scalar and per-token temperature for chunked_lm_head_logprob(deterministic=False) by @yyihuang in #8436
• perf(cake_mega_moe_v3): faster TP4 EP1 T4096, TP4 EP4 and TP8 EP8 T128 rank programs on sm_103a by @yyihuang in #8458
• perf(cake_gdn): sector-complete FP32 state I/O in SM107 full-DV prefill (up to 1.17x) by @yyihuang in #8461
• perf(cake_gdn): TMA-staged state for sm_107a nontranspose GDN decode, B>=32 (up to 1.11x) by @yyihuang in #8462
• feat(moe_ep): add Rubin offline variant selection by @akaashrp in #8427
• perf(cake_tp_moe_finalize_rmsnorm): route B200 TP8 tokens 640-1008 to the cluster-pair token-owner program by @yyihuang in #8463
• perf(cake_dense_projection_det_gemm, cake_dense_projection_gemm): programmatic dependent launch on every program, safe by default by @yzh119 in #8389
• perf(cake_fmha): aligned hd256 decode cluster-of-8 program for sm_100a / sm_103a by @yyihuang in #8418
• perf(cake_grouped_fp8_gemm): route six UE8M0 sm_103a layouts to faster swap-AB programs by @yyihuang in #8467
• perf(cake_fmha): cheaper setup for the length-aware BF16 decode programs by @yyihuang in #8469
• feat(cake_kda_det): with grad, in-kernel Q/K L2 norm, use_qk_l2norm_in_kernel, single-rank (#7817) [stacked] by @yyihuang in #8277
• perf(cake_rmsnorm_silu_gate_train): non-deterministic short forward computes the SiLU gate before the sum of squares on SM100/SM103 by @yyihuang in #8465
• perf(cake_kda): with grad, cached FFI launch and per-form fence in the non-deterministic single-rank package by @yyihuang in #8455
• perf(cake_gdn_kcp): plain split views of packed Q/K/V get contiguous dq / dk / dv in the non-deterministic KCP backward by @yyihuang in #8435
• perf(cake_gdn_det): run serial adjoint scan beside tail_g3 on two streams on B200/GB300 by @yyihuang in #8471
• perf(cake_kda): BF16 KDA prefill helper-pool bin-mix, mixed-length M64 split and FP32-state ping-pong chain corrections by @yyihuang in #8460
• perf(cake_deepgemm): faster SM120 paged FP8 MQA logits routes for 20 RTX 5090 and 24 RTX PRO 6000 shapes by @yyihuang in #8473
• perf(cake_dsv4): 256-bit O stores in fp8_lowhead_one_partition on sm_103a by @yyihuang in #8476
• perf(cake_deepgemm_dense_mqa): source consumer release on the sm_103a fp8:qle2048:short program and its stage bucket by @yyihuang in #8475
• perf(cake_fmha): add BF16 runtime-size causal paged context programs by @yyihuang in #8478
• perf(cake_msa_icp_indexer): sm_107a rows-32 list scorer triggers the list publisher after the epilogue by @yyihuang in #8481
• feat(cake_kda_det_kcp): with grad, in-kernel Q/K L2 norm on the KCP route (#7817) [stacked on #8255] by @yyihuang in #8259
• perf(cake_dense_projection_gemm): measured 216-SM sm_107a K1 routes: 192-wide 4096 -> 1024 FP32 and six more deterministic token counts by @yyihuang in #8438
• fix(cake_causal_conv1d_det): opt-in non-cooperative KCP backward (cooperative=False) that does not wait for kernels on other streams by @yyihuang in #8459
• fix(moe_ep): fix Rubin tuning cache reuse and schedule sweeps by @akaashrp in #8477
• perf(cake_alpha_moe): fused router ranks staged values with f32 compares in the large rebase-batch units by @yyihuang in #8484
• perf(cake_rmsnorm_det): with grad, H=256 column-split dw reduce on SM100/SM103, SM103 lean compensated ring for large T by @yyihuang in #8439
• perf(cake_causal_conv1d): keep the non-deterministic backward one wave beside a concurrent kernel by @yyihuang in #8429
• perf(cake_causal_conv1d): faster single-rank non-deterministic backward on sm_107a by @yyihuang in #8450
• perf(cake_kimi_k3_attn_res): token-queue programs for more B200, B300 and GB300 shapes by @yyihuang in #8489
• perf(cake_fmha): runtime-size FP8 paged context programs for GQA packs 5, 6, 7, 12, 16 by @yyihuang in #8486
• perf(cake_dense_projection_gemm): sm_103a router weight-gradient head/tail program with the A parts in tensor memory by @yyihuang in #8444
• perf(cake_dense_projection_det_gemm): add GB200 / GB300 T-exact row rules for the q_b input gradient and the 17408 x 5120 weight gradients by @yyihuang in #8491
• fix(cake_dsv4): stage misaligned caller out on every route with wide O stores by @yyihuang in #8493
• perf(cake_lm_head_loss): zero-gradient row skip and short-chunk TMA dW cast on sm_107a by @yyihuang in #8492
• perf(cake_softmax): low-latency shared-tile kernels for grids of at most two waves by @yyihuang in #8449
• fix(cake_rmsnorm): deterministic=False runs the deterministic forward and moves backward windows for dw accuracy by @yyihuang in #8253
• perf(cake_mhc_det): with grad, dual-layout fn split for the activation-checkpoint recompute on sm_107a by @yyihuang in #8488
• perf(cake_dense_projection_gemm): launch the K1 programs through bound argument lists by @yyihuang in #8456
• perf(cake_moe_finalize_allreduce_fusion): quad-route programs for batches of up to 142 tokens at world size 2 by @yyihuang in #8503
• perf(cake_rmsnorm_silu_gate_train): short forward up to 12 rows per SM on SM100/SM103 by @yyihuang in #8505
• chore(cake_kda_det): with grad, text-only re-render, fallback replay launch count, drop 4 unreachable builds by @yyihuang in #8500
• perf(cake_deepgemm_mega_mhc): weight-staging tail routes for 3- and 4-wave sm_107a launches on R200 by @yyihuang in #8507
• benchmark(cake_dense_projection_gemm): router SOTA = the fastest library arm that meets the router budget by @yyihuang in #8508
• perf(cake_grouped_fp8_gemm): relax the prologue cluster arrive of the sm_103a 64-row-block UE8M0 program by @yyihuang in #8511
• perf(cake_causal_conv1d): sm_107a C=1024 channel-band programs for the non-deterministic backward by @yyihuang in #8510
• perf(cake_mok_bf16): vectorized shared-memory MXFP8 dispatch quantize in the sm_107a forward (#7821) by @yyihuang in #8506
• perf(cake_mega_moe_v3): faster TP4 / TP8 EP1 T128, T512, T1024, TP8 EP1 T4096 and TP4 EP4 T4096 rank programs on sm_103a by @yyihuang in #8515
• perf(cake_lm_head_loss): zero-gradient row skip for the log-probability backward with a temperature by @yyihuang in #8514
• perf(cake_softmax): low-latency shared-tile kernels at every wave count on sm_100 by @yyihuang in #8513
• fix(cake_alpha_moe): keep all-NaN rows in range in the generic fused router by @yyihuang in #8519
• fix(quantization): make the FP8 NVFP4 TMA quantize path encode and cover whole column chunks by @almightychang in #6276
• docs(sampling): document per-row top_k / top_p / min_p tensors when indices is set by @Wint3rNight in #5324
• feat(cake_causal_conv1d): opt-in SM reserve for the non-deterministic KCP autograd backward by @yyihuang in #8501
• perf(cake_dense_projection_det_gemm): run the half-tail TMA-store program on the sm_103a 5120 x 17408 weight gradients at T 16231 / 16384 by @yyihuang in #8522
• fix(sampling): remove CUDA 11.8 FlagHeads fallback by @yonglai in #5124
• perf(cake_tp_moe_finalize_rmsnorm): route B200 TP8 tokens 633-639 to the cluster-pair token-owner program by @yyihuang in #8524
• ci: re-enable H100 tests on pull requests by @righthandabacus in #8368
• perf(cake_gdn_cp): int32 cu_seqlens Stage-1 program for FP16 CP prefill on sm_107a (up to 1.05x) by @yyihuang in #8523
• perf(cake_tinygemm2_wgmma): sm_90a routing extended to input features above 14336 by @yyihuang in #8529
• Guard redundant same-value stores in norm, decode, and cascade kernels by @jiazhihao in #5740
• cake batch DeepGEMM FP8: 256-byte L2 promotion on the A and B tensor maps of two masked programs by @yyihuang in #8528
• perf(cake_dense_projection_gemm): 212-SM sm_107a router weight-gradient windows run the head/tail program with the A parts in tensor memory by @yyihuang in #8517
• [SDSA] Add the SDSA (Simplified DSA) indexer for Blackwell, with BF16 scores on other GPUs by @netanel-haber in #8374
• feat(moe_ep): build the NVLink one-sided Lamport transport for SM107 by @arpera in #8419
• perf(cake_gdn_det_kcp): chain-aware walk widths in the deterministic KCP GDN package by @yyihuang in #8538
• fix(rope): guard the non-RoPE tail and honor V cache strides in the paged-KV append by @0z5a in #7494
• perf(cake_rmsnorm_silu_gate_train): short forward up to 10 rows per SM and one-cluster short backward on SM107 by @yyihuang in #8540
• perf(cake_fmha): L2 cache-policy hints for the DCP spec-decode E4M3 hd256 balanced programs on sm_100a / sm_103a by @yyihuang in #8504
• test: allow appended options after CAKE request-order arguments by @righthandabacus in #8531
• perf(cake_dsv4): route short FP8 64-head decode to fp8_lowhead_h64 with PDL on sm_100a/sm_103a by @yyihuang in #8543
• perf(cake_gdn_kcp): chain-aware value widths for the state walks of the non-deterministic KCP package by @yyihuang in #8521
• fix(autotuner): serve lookups above the largest tuned bucket from it by @ji24077 in #7626
• ci: use quack-kernels compatible with the supported CUTLASS DSL by @righthandabacus in #8534
• fix(jit): keep compatibility re-exports out of strict cache closures by @righthandabacus in #8532
• ci: resolve CUTLASS DSL libraries before syncing package metadata by @righthandabacus in #8533
• fix(gdn): fp32 MTP pool scatter against real serving pools by @brandonfzhang in #7882
• perf(qsa): speed up BF16, FP8 and NVFP4 on SM80-SM90 by @lesj0610 in #8423
• fix(sampling): align indexed top-k and speculative metrics by @taking-lying-flat in #8002
• fix(quantization): reject unsupported FP4 quantization targets by @YaoleWu in #7987
• perf(sampling): split rows across thread block clusters in rejection samplers on SM90 by @SamMausberg in #5617
• Remove the CuTe-DSL quantize kernel entry points from the public API by @Vinnie6167 in #8391
• perf(cake_vsa): shared-selection block-64 balanced program on SM100/SM103 by @yyihuang in #8525
• fix(gemm): give each CUDA stream its own GEMM workspace by @gau-nernst in #8221
• perf(cake_gdn): route sm_107a FP32 T=1 nontranspose decode B 21-31 to the TMA-state body (up to 1.17x) by @yyihuang in #8547
• perf(cake_mok_bf16): vectorized shared-memory MXFP8 dispatch quantize in the sm_107a backward (#7821) by @yyihuang in #8546
• perf(cake_msa_icp_indexer): fused prefill scoring with local Top-16 lists for TP2/TP4 x P128/P256 by @yyihuang in #8502
• perf(cake_dense_projection_gemm): sm_103a router weight-gradient head/tail windows at hidden 4096 and 6144 by @yyihuang in #8518
• perf(cake_kda): per-warp TMA final-state store and lower host launch cost in BF16 KDA prefill by @yyihuang in #8544
• perf(cake_mega_moe_v3): faster TP4 EP1 and TP8 EP8 T4096 rank programs on sm_103a by @yyihuang in #8553
• feat(cake_kda): with grad, raw forget-gate mode in the non-deterministic single-rank and context-parallel packages by @yyihuang in #8550
• perf(cake_kimi_k3_attn_res): faster programs for more B200, B300 and GB300 shapes by @yyihuang in #8554
• tests(cake_kda_det_kcp): rendezvous spawned ranks through a file store; skip uncovered GPUs without raising by @yyihuang in #8516
• perf(cake_tinygemm2_wgmma): sm_90a multi-wave U4 shapes routed to the two-CTA-per-SM packed programs by @yyihuang in #8555
• perf(cake_dsv4): PDL-free twin of the FP8 H128 persistent program on SM100/SM103 by @yyihuang in #8557
• perf(cake_rmsnorm_silu_gate): with grad, short forward up to 59 rows per SM on SM107 by @yyihuang in #8556
• perf(cake_dense_projection_gemm): cache the router entry points' calls by their own tensors by @yyihuang in #8539
• fix(norm): preserve row strides in CUDA LayerNorm by @yuhengcai1 in #8378
• fix(norm): align RMSNorm cluster smem budget with the cp.async threshold by @sout30052 in #5729
• build: post-process the flashinfer-python wheel so release wheels stay under PyPI's 100 MB limit by @aleozlx in #8011
• perf(cake_causal_conv1d_det): with grad, row-group backward program for 264 <= C < 512 on SM107 by @yyihuang in #8541
• test(cake_gdn_kcp): pin the architecture the backend passes to the #7 walks by @yyihuang in #8559
• Add swap-AB mode and bit-exact testing to split CuTe DSL MoE by @zianglih in #7290
• fix(cake_gdn): keep the segment transfer plane in the full saved set of the KCP GDN package by @yyihuang in #8542
• perf(cake_grouped_fp8_gemm): 256-byte L2 promotion on the A and B tensor maps of the deep-K N256 program by @yyihuang in #8520
• perf(cake_rmsnorm): launch the non-deterministic column tasks at once and run the deterministic pair in two GB300 backward windows by @yyihuang in #8577
• perf(cake_deepgemm_mega_mhc): weight-staging tail routes at two and three waves on sm_107a by @yyihuang in #8571
• perf(cake_msa): blackwell_msa fused K-major prefill, qpair BF16 decode, FP8 fuseq8 by @yyihuang in #8574
• fix(cake_dense_projection_gemm): run the deterministic program for four FP32 sm_107a K1 rows at large token counts on 216-SM parts by @yyihuang in #8509
• perf(cake_gdn_det): batch-of-eight elected wavefront release for the dhu scan on R200 by @yyihuang in #8587
• perf(cake_dense_projection_det_gemm): store the indexer_q weight gradient's stream-K tiles through the TMA store at T 16172 / 16231 by @yyihuang in #8584
• perf(cake_dcp): run the fold-free planner program for near-grid E4M3 head_dim-128 launches by @yyihuang in #8586
• perf(cake_msa_icp_indexer): faster prefill list-to-carrier merge by @yyihuang in #8593
• perf(cake_mega_moe_v3): faster TP4 EP4 and TP8 EP8 T4096 rank programs on sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8594
• perf(cake_gdn_cp): row-split sm_107a CP state fixup for 4, 8, 16 and 32 parallel states (up to 1.08x) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8572
• fix(cake_gdn_det_kcp): keep M_seg in the full saved set at P > 1 and count the whole plane in the saved byte query by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8585
• perf(cake_vsa): per-architecture block-64 balanced route on SM100/SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8588
• perf(cake_mhc_det): with grad, collapse pass reads the mapping-pass input from L2 on sm_100a / sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8582
• Collect continuous host, GPU and cgroup test telemetry by @righthandabacus in https://github.com/flashinfer-ai/flashinfer/pull/8589
• perf(cake_nvfp4_mla_decode): two-split combine for short GB300 decodes by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8596
• fix(cake_rmsnorm_det): fixed-order dw for the smallest token counts of the SM100/SM103 backward by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8512
• feat(cake_fp8_rowwise_gemm_allreduce): SM103 fused FP8 GEMM + all-reduce for TP2/4/8 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8605
• perf(cake_fmha): keep power-of-two query lengths on the JIT module for long causal BF16 context on sm_100a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8608
• test(cake_dense_projection_gemm): key the router head/tail program test on the part's SM count by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8603
• perf(cake_mok_bf16): vectorized MXFP8 dispatch quantize in the sm_107a recompute (#7821) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8604
• perf(cake_kimi_k3_attn_res): CLC lookahead token-queue programs for one B200 and one GB300 shape by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8609
• perf(cake_gdn_kcp): cheaper eager host path for non-deterministic KCP GDN autograd by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8600
• fix(cake_hopper_msa): exact signed-zero tie-break in the top-k block-select programs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8606
• perf(cake_dense_projection_gemm): run the split program at measured token ranges on 212-SM sm_107a parts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8530
• perf(cake_softmax): issue register-tile loads before the CTA barrier in the shared-tile kernels by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8597
• perf(cake_dense_projection_det_gemm): store the 5120 -> 17408 input gradient's overlapped tiles through the TMA store at T 16231 / 16384 on sm_100a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8614
• perf(cake_gdn_det): packed-triangular forward WY program at five CTAs per SM on B200 and GB300 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8615
• perf(cake_deepgemm): faster SM120 paged FP8 MQA logits routes for 4 RTX 5090 and 12 RTX PRO 6000 shapes by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8621
• msa_ops ICP indexer: short prefill windows up to 384 tokens on B200 TP2 page 128 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8624
• fix(cake_gdn_kcp): byte queries of the non-deterministic KCP GDN package match what the calls bind by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8623
• feat(batch_invariant): batch-invariant quantizers: mxfp8_quantize, mxfp4_quantize and nvfp4_quantize(batch_invariant=True) by @bkryu in https://github.com/flashinfer-ai/flashinfer/pull/8565
• perf(cake_lm_head_loss): counted temperature dz recompute under the zero-gradient row skip by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8612
• perf(cake_fmha): add NVFP4-KV runtime-size paged context programs for sm_100a and sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8619
• perf(prims-ts): faster D576 and D512 (DSv4) sparse MLA decode on SM103/SM107 by @PerkzZheng in https://github.com/flashinfer-ai/flashinfer/pull/8322
• perf(cake_rmsnorm_silu_gate_train): L2-prefetching mid forward on SM100 and SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8622
• perf(cake_mega_moe_v3): faster TP4 / TP8 EP1, TP4 EP4 and TP8 EP8 T4096 rank programs on sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8627
• perf(cake_rmsnorm_train): run the deterministic package's pairs in more backward windows by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8626
• perf(cake_kda_det_kcp): shorter host prologue of the in-kernel q/k L2 norm backward by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8552
• perf(cake_softmax): issue register-tile loads before the CTA barrier in the 8-CTA low-latency kernels by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8617
• perf(cake_kda): per-warp TMA final-state store for short many-task turnover launches on sm_100a in BF16 KDA prefill by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8628
• perf(cake_dense_projection_gemm): measured K1 token coverage on 148-, 212- and 216-SM parts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8595
• perf(cake_mxfp8_megamoe_ep16): CuTe-DSL programs on the CUDA backend's cross-rank protocol by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8631
• perf(cake_kda_det_train): glm_intra v30 runs the fused Q/K L2 norm's GLM intra stage on sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8625
• feat(cake_moe_grouped_gemm): fuse the fp32 weight-gradient accumulate into main_grad on sm_100a, sm_103a and sm_107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8602
• perf(cake_alpha_moe): NVFP4 fused batch kernel quantizes its activation per lane by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8632
• perf(cake_msa_icp_indexer): tagged decode merge at TP4 polls its rows before it waits by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8633
• fix(cake_kda_det): with grad, zero the next group's qn / kn rows before the batched l2n replay by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8629
• perf(cake_dense_projection_det_gemm): add the sm_107a 4096 -> 1024 fp32 weight-gradient key at T 16384 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8635
• perf(cake_causal_conv1d_det): with grad, 32-vector channel tiles for narrow single-tile rows by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8611
• perf(cake_dense_projection_gemm): FFMA router weight gradient up to 192-256 tokens on sm_100a/sm_107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8578
• perf(cake_all_gather_matmul): SM100a last-pass N-split twin for 1025-row bf16 K-major calls at world size 8 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8645
• perf(cake_tp_moe_finalize_rmsnorm): route R200 TP4 tokens 640-864 to a tail-fold token-owner program by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8643
• perf(cake_kda_det): with grad, cached FFI entry and raw-stream launch in the deterministic single-rank package by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8630
• perf(cake_kda): with grad, reduce the raw gate's parameter gradients in FP64 in the non-deterministic packages on sm_100a and sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8620
• fix(cake_dense_projection_gemm): run 4096 -> 64 BF16 output deterministic from 65536 tokens on 148-SM sm_100a parts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8636
• perf(cake_kda): per-warp TMA final-state store for long-task bin-mix launches on sm_100a in BF16 KDA prefill by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8654
• perf(cake_deepgemm_mega_mhc): weight-staging routes for the 16-split two-wave non-deterministic band on sm_107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8651
• perf(cake_rmsnorm_silu_gate): L2-prefetching mid forward on SM107 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8656
• perf(cake_gdn_kcp): with grad, full-save forward writes the output and the tapes in one walk by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8644
• perf(cake_dense_projection_det_gemm): store the 12288 -> 4096 forward's overlapped tiles through the TMA store at T 16231 and 16384 on sm_100a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8660
• perf(cake_dense_projection_gemm): FFMA router input gradient up to 64-128 tokens on sm_100a/sm_107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8618
• perf(cake_dsv4): issue the SM107 FP8 64-head multi-tile M64 QK at K=64 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8661
• perf(cake_rmsnorm): blocked FP32 dw sums for the large-T GB300 and R200 backward by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8655
• perf(cake_dense_projection_gemm): two-split K1 weight gradient for the wide FP32 rows on 212-SM sm_107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8657
• perf(cake_kda_det): glm_bwd_prep v11_g8 runs the KCP saved-rstd backward's replay prep on sm_100a / sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8652
• perf(cake_dense_projection_det_gemm): store the 5120 -> 17408 input gradient's overlapped tiles through the TMA store at T 16231 and 16384 on sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8667
• perf(cake_dsv4): K=64 FP8 MMA program for the SM107 128-head uniform-gather persistent route by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8668
• perf(cake_gdn): packed-triangular forward WY program at five CTAs per SM on B200 and GB300 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8665
• perf(cake_comm): hidden-size per-destination thresholds and SM103 eight-rank cross-band plan for QKV exchange by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8649
• perf(cake_gdn_kcp): with grad, graph-replay checkpoint recompute hands its tapes to the backward by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8638
• fix(cake_dense_projection_gemm): plan cached router launches again after a router-table change by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8663
• perf(cake_mega_moe_v3): faster TP8 EP1 T32 / T128 / T512 / T1024 rank programs on sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8669
• fix(rope): stop near-tie grid splits on large prefill; test the forced schedules by @jimmyzho in https://github.com/flashinfer-ai/flashinfer/pull/7831
• perf(cake_dense_projection_gemm): projection entry points go from their key to the bound launch by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8650
• perf(cake_dense_projection_gemm): sum the router's split-K partials with a fixed-order program in the bound launch chain by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8664
• perf(cake_dsv4): issue the sm_107a FP8 low-head prefill MMAs at K=64 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8670
• [test] KDA decode call-shape equivalence + kernel-census + fp32-rejection tests (#6560) by @brandonfzhang in https://github.com/flashinfer-ai/flashinfer/pull/8052
• fix(quantization): preserve signed zero in E2M1 fallback by @anencore94 in https://github.com/flashinfer-ai/flashinfer/pull/8397
• perf(cake_msa): blackwell_msa paged BF16 8-token decode at 64/4 heads, batch 64/128 on the GQA-16 union program by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8672
• feat(moe_ep): post-FC2 routing weights in the SM107 NVFP4 MegaMoE tuner by @jdebache in https://github.com/flashinfer-ai/flashinfer/pull/8013
• fix(norm): mixed-architecture dispatch compatibility guard by @XFDG in https://github.com/flashinfer-ai/flashinfer/pull/8474
• perf(cake_rmsnorm_silu_gate_train): L2-prefetching extra-long forward on SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8673
• perf(cake_gdn): run the split adjoint scan beside the group-of-three tail on B200 and GB300 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8675
• perf(cake_kimi_k3_attn_res): L2 first-chunk prefetch and CLC token-broadcast programs for two B200 and five B300 shapes by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8676
• fix(sampling): isolate softmax partial reductions across streams by @zupengwang in https://github.com/flashinfer-ai/flashinfer/pull/4692
• feat(gdn): add SM120 cuteDSL ReplaySSM GDN MTP decode kernels by @Tom-Zheng in https://github.com/flashinfer-ai/flashinfer/pull/7536
• perf(quantization): select the NVFP4 TMA quantize kernel by input size by @almightychang in https://github.com/flashinfer-ai/flashinfer/pull/6691
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM103 2048- and 4096-token rows: fused quantization + reset and per-CTA privatized routing by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8674
• feat(kda): indexed state pool (ssm_state_indices) for the small-BH prefill backend by @longlee0622 in https://github.com/flashinfer-ai/flashinfer/pull/8287
• perf(cake_dsv4): SM107 short-query decode launches keep the K=64 uniform program past 128 rows by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8679
• perf(cake_mega_moe_v3): faster TP4 EP4 T512 / T1024 rank programs on sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8682
• feat(cake_grouped_fp8_gemm): block-scaled contiguous grouped FP8 GEMM on SM107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8683
• perf(cake_dense_projection_det_gemm): plan the 17408 -> 5120 forward in 12-column-tile raster groups at T 16231 / 16384 on sm_100a and sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8686
• feat(prefill): teach the cuDNN ragged path sliding window, attention sinks and scalar scales by @Anerudhan in https://github.com/flashinfer-ai/flashinfer/pull/5286
• fix(cake_dense_projection_gemm): route K1 entries that fail the accuracy rule to the deterministic program by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8671
• ci(docker): split the CI image pip install into torch, deps and cuDNN layers by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/7899
• perf(cake_lm_head_loss): per-architecture floor of the zero-gradient row skip, lighter log-prob backward host path by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8687
• feat(cake_nvfp4_ptilemax_attention): add the SM103 NVFP4 per-tile-max attention as a separate API by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8693
• perf(cake_grouped_fp8_gemm): earlier next-tile prefetch and two more small-block UE8M0 routes on SM100a/SM103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8694
• feat(quantization): validate-once launcher for the NVFP4 KV materializer by @xuanzic in https://github.com/flashinfer-ai/flashinfer/pull/8057
• perf(cake_moe_finalize_allreduce_fusion): deferred two-shot programs with a shared expert at world size 4 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8697
• feat(cake_batch_deepgemm_fp8): masked BLOCK_N batch DeepGEMM FP8 on SM107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8688
• perf(cake_kimi_k3_fp8_projection): SM100a early first-stage loads in small-M decode programs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8690
• perf(cake_rmsnorm_silu_gate_train): L2-prefetching extra-long forward on SM107 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8701
• perf(cake_gdn_cp): row-split sm_107a CP state fixup for 14 to 48 parallel states (up to 1.10x) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8700
• perf(cake_gdn_det): packed-triangular forward WY program at five CTAs per SM on R200 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8695
• perf(cake_mhc_det): with grad, collapse pass reads the mapping-pass input from L2 on sm_107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8698
• perf(cake_dsa_indexer_det): ticket scheduling on sm_107a, no fill kernel in eager entry by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8691
• perf(cake_dense_projection_det_gemm): plan the 5120 -> 17408 weight gradient with the half-tail TMA-store program at T 16231 / 16384 on sm_100a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8703
• feat(cake_dense_projection_gemm): two-split K1 for 4096 -> 128 FP32 weight gradients on 152-SM and 212-SM parts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8680
• perf(cake_rmsnorm): fixed-claim column-task backward for the smallest token counts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8704
• ci: overlap PR AOT provider builds within one job by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/7897
• ci: share the GPU compile cache through S3, written only by scheduled main runs by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/7016
• Fix/ulysses lowp provider targets by @cindyzxq in https://github.com/flashinfer-ai/flashinfer/pull/8487
• perf(cake_mega_moe_v3): faster TP4 EP1 T128 / T512 / T1024 rank programs on sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8706
• test: fix architecture guards and mocked backend contracts by @akaashrp in https://github.com/flashinfer-ai/flashinfer/pull/8647
• perf(cake_mhc): with grad, the non-deterministic collapse pass reads the mapping-pass input from L2 on sm_100a / sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8677
• perf(cake_dsv4): route SM107 FP8 8-head prefill and 8/16/32-head two-wave SWA decode to the M64 programs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8699
• test(cake_grouped_fp8_gemm): numerical cases for the new swap-AB C1 layouts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8705
• perf(cake_kda): per-warp TMA final-state store for the bounded direct turnover launches in BF16 KDA prefill (sm_100a, sm_103a) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8711
• perf(cake_batch_deepgemm_fp8): early exit for an all-empty batch in the SM100a N=7168, K=2048 serving program by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8709
• feat(checkpointing_ssu): accept strided ring_start and prev_num_accepted by @tomeras91 in https://github.com/flashinfer-ai/flashinfer/pull/8548
• perf(cake_batch_deepgemm_fp8): relaxed cluster rendezvous for the swap-AB BLOCK_M 128 program and a lean-epilogue N=128, K=512 band program on SM103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8708
• perf(cake_dsv4): SM107 FP8 H128 K=64 uniform-gather program double-buffers its output staging by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8713
• fix(cake_dense_projection_gemm): route router windows that fail the accuracy rule to the deterministic program by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8689
• perf(cake_dsv4): route SM107 FP8 64-head decode rows of two to four sparse tiles to the M64 body before the half wave by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8715
• Add the template keyword explicitly. by @cyk2018 in https://github.com/flashinfer-ai/flashinfer/pull/4862
• ci: share release/nightly build pipeline via a reusable workflow by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/7898
• perf(cake_mok_bf16): Gluon MXFP8 weight quantizer, one launch for gate/up/down by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8717
• perf(cake_dense_projection_gemm): three accumulation chains for the one-split sm_107a FP32 weight gradients by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8685
• ci: recommend test scopes for bot run commands by @righthandabacus in https://github.com/flashinfer-ai/flashinfer/pull/6814
• perf(cake_dense_projection_det_gemm): plan the 17408 -> 5120 and 16384 -> 6144 forwards with the row-half-release program at T-exact keys by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8719
• fix(moe): pad_trtllm_moe_weights helper plus valid_* assert messages and the pad-and-declare contract for unaligned MoE shapes by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/8401
• fix(comm): retain DLPack metadata and allocations for tensor views by @akaashrp in https://github.com/flashinfer-ai/flashinfer/pull/8601
• feat(quantization): encode and expand CSF block scales by @lesj0610 in https://github.com/flashinfer-ai/flashinfer/pull/8454
• fix(b12x): honour deterministic_output in the fused MoE decode heuristic and the tuner validation by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/8403
• fix(prims_ts): pad MXFP8 activation scales in the MXFP4xMXFP8 MoE runner by @yekerr in https://github.com/flashinfer-ai/flashinfer/pull/6607
• fix(moe): reject non-contiguous hidden_states in Prims-TS BF16 MoE by @yekerr in https://github.com/flashinfer-ai/flashinfer/pull/6608
• perf(cake_batch_deepgemm_fp8): 256-byte L2 promotion for the SM103a N=4096, K=7168 band problems of at most two M tiles per group by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8720
• perf(cake_msa): blackwell_msa paged FP8-Q prefill (1 query token, 64/4 heads, 1 page) on a new union program by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8723
• perf(cake_lm_head_loss): sm_107a ND backward uses the sm_103a GEMM tile rules by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8724
• fix(moe): default cutlass_fused_moe to the unfused finalize under torch deterministic mode by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/8442
• fix(moe): keep unified MoERunner persisted autotune keys rank-invariant, with a cross-process cache-hit test by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/8581
• docs(sampling): explain why per-row filter params are keyed by row_idx by @0z5a in https://github.com/flashinfer-ai/flashinfer/pull/5346
• test(moe): cover routing_replay_out for MXFP8 block-scale TRTLLM MoE by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/8398
• test(moe): pin W4A16 CuTe-DSL MoE batch invariance by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/8399
• perf(cake_mega_moe_v3): faster TP8 EP1 T128 / T512 / T1024 rank programs on sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8731
• fix(tests): copy CUDA graph function handles before retaining them by @righthandabacus in https://github.com/flashinfer-ai/flashinfer/pull/8729
• perf(cake_mhc): with grad, the non-deterministic collapse pass reads the mapping-pass input from L2 on sm_107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8732
• fix(cake_dsv4): reject FP8 8-head prefill items wider than 1152 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8733
• fix(sm12x): correct packed scale layouts and kernel reuse by @chengmingz-nv in https://github.com/flashinfer-ai/flashinfer/pull/7704
• perf(cake_kda): overlap the short tail of the BF16 KDA prefill mixed-length range split (sm_100a, sm_103a) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8738
• perf(cake_mxfp8_megamoe_ep16): sm_103a CuTe-DSL fused programs read the peer tables from shared memory by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8735
• perf(cake_dense_projection_det_gemm): plan the sm_103a 17408 -> 5120 and 12288 -> 4096 forwards with the row-half-release program at T-exact keys by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8737
• perf(cake_minimax_h3): staged norm and SF-ring GEMM for one and two NVFP4 destinations on SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8739
• perf(cake_mhc): with grad, slab-order bands for the 152-SM sm_103a board at hidden 7168 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8662
• add-frost-combine-moe by @yanqinz2 in https://github.com/flashinfer-ai/flashinfer/pull/8692
• perf(cake_fmha): interleave the tail units of runtime-size BF16 context launches by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8746
• perf(cake_rmsnorm_silu_gate_train): one-head inputs take the packed-rows L2 prefetch by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8748
• fix(cake_deepgemm_dense_mqa): honour clean_logits in the row-range split; B200 raw-variant test points by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8742
• perf(cake_dsa_indexer_det): split few-query calls so each split holds at most top_k keys on sm_100a and sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8725
• perf(cake_dense_projection_gemm): compensated FP32 SIMT router programs in few-token windows by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8707
• perf(cake_kda): TMA entering-state load for short many-task turnover launches in BF16 KDA prefill (sm_100a) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8750
• perf(cake_fmha): four-stage K ring for program 0 of the E4M3 head_dim-128 balanced DCP family (sm_100a, sm_103a) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8751
• perf(cake_dsv4): refine the SM107 FP8 64-head decode route edges by complete-tile count by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8753
• fix(cake_gdn_det_kcp): send only written data in the backward boundary-adjoint all-to-all by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8741
• perf(cake_mega_moe_v3): faster TP4 EP1 T512 / T1024 and TP8 EP8 T1024 rank programs on sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8756
• feat(cake_dsa_indexer_det): opt-in FP8 (e4m3) form by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8749
• fix(ci): harden sccache fallback handling by @cindyzxq in https://github.com/flashinfer-ai/flashinfer/pull/8001
• perf(cake_lm_head_loss): make the log-prob backward's gradient buffers behind its recompute launch by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8762
• perf(cake_fmha): FP8 paged-context runtime-size programs for the non-causal, LSE and sink forms by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8759
• perf(cake_dense_projection_gemm): restore the K1 weight-gradient routes that meet the deterministic package's absolute FP64 budgets by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8743
• perf(cake_dense_projection_det_gemm): plan the 4096 -> 12288 input gradient with the row-half-release program at T-exact keys by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8765
• perf(cake_gdn_det_kcp): fold the output program into the forward walks of the deterministic KCP package by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8755
• perf(cake_kimi_k3_fp8_projection): SM103a MMA-warp TMEM allocation twin of the TMA-store GEMM for TP1 kv_b M 513..1024 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8767
• perf(cake_causal_conv1d_det): SM-wide row-group backward for 264 <= C < 512 on SM100 and SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8761
• perf(cake_gdn): packed-triangular forward WY program at five CTAs per SM on R200 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8776
• perf(cake_minimax_h3): tile-lane MXFP8 fused QKV GEMM program on SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8775
• perf(cake_gdn_kcp): opt-in light_akk save mode keeps Akk tape to skip the solve by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8754
• fix(cake_dsv4): fall back from the SM107 K=64 program without oversized shared memory by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8777
• perf(cake_kda): one-helper owner/helper hybrid build for equal-length 37-48-task launches in BF16 KDA prefill (sm_100a) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8764
• perf(cake_dense_projection_gemm): restore the SIMT and head/tail router windows on 152-SM and 212-SM parts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8730
• perf(cake_moe_grouped_gemm): use the L2 evict-first 512-wide dgrad tile on sm_100a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8773
• perf(cake_deepgemm): route two SM120 batch-128 64K-context decode steps to faster programs (RTX PRO 6000 h64 p64 n4, h64 p64 n3) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8786
• perf(cake_rmsnorm_silu_gate): with grad, FP32 two-sum fold on SM103 from 700 rows per SM by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8785
• fix(cake_fmha): plan small-M splits on the launch stream's partition by default, capture-safe (#8641); count profiled launches between sentinels (#8472) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8789
• fix(cake_gdn_det_kcp): count the full-mode forward tapes without a saved set in the workspace query by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8783
• perf(cake_kimi_k3_attn_res): L2 first-chunk prefetch programs for three B200, eleven B300 and eleven GB300 shapes by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8791
• perf(cake_dcp): run near-grid E4M3 head_dim-128 launches with 4+ query rows on program 0 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8797
• perf(cake_dense_projection_gemm): one split of 64-wide tiles for the FP32 K1 weight gradient 4096 -> 32 / 128 on 152-SM parts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8760
• perf(cake_batch_deepgemm_fp8): SM107a (B, expected_m) cell table for three masked geometries by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8798
• perf(cake_dsa_indexer_det): split few-query calls so each split holds at most top_k keys on sm_107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8802
• perf(cake_dsa_indexer_det): run the FP8 l6 program for wide-unit decisions by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8781
• chore(cake_causal_conv1d_det_train): record the published generator revision by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8804
• perf(cake_dense_projection_gemm): plain FP32 SIMT router windows on 148-SM sm_100a / sm_103a parts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8763
• fix(cake_rmsnorm_train): end the sm_103a H2048 det-pair backward windows below their dw budget breach by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8810
• perf(cake_grouped_fp8_gemm): SM107a single-run BLOCK_N 256 below 32 K blocks by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8809
• perf(cake_dense_projection_gemm): restore the K1 weight-gradient routes on 216-SM sm_107a parts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8782
• fix(cake_dense_projection_gemm): keep workspace buffers alive for captured CUDA graphs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8812
• perf(cake_vsa_sm120): run vsa_sm120_blk64 backend on Cake kernels by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8799
• fix(cake_gdn_det_kcp): make the byte queries of the deterministic KCP package match what the calls bind by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8808
• fix(kda): double-buffer update_ready in the CuTe DSL decomp chain kernel by @alphabetc1 in https://github.com/flashinfer-ai/flashinfer/pull/8684
• test: restore flashinfer modules after logging replay tests by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/8437
• fix(jit): remove NVFP4 kernel dependency on the quantization API by @righthandabacus in https://github.com/flashinfer-ai/flashinfer/pull/8728
• perf(cake_kda): FP32 state decay after the full state hand-off in the N16 checkpoint-TMA direct build of BF16 KDA prefill (sm_100a + sm_103a) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8814
• perf(cake_lm_head_loss): two-slice counted dX for one-slice chunks under the zero-gradient row skip (sm_100a, sm_103a) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8796
• perf(cake_dense_projection_det_gemm): measured split counts for the router weight gradient at H 5120 / 6144 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8816
• perf(cake_comm): fused multiply-add epilogue defaults on SM100 / SM103 and SM100 eight-rank cross-band plan for QKV exchange by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8800
• perf(cake_dsv4): four-stage K/V ring program for SM107 FP8 64-head multi-tile M64 launches by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8818
• perf(cake_batch_deepgemm_fp8): prefetch masked_m into L2 on sm_100a swap-AB by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8819
• perf(cake_kda_det_kcp): v2 affine merge, offset walks and BF16x3 transfer walk, bitwise identical to v1 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8803
• chore(codeowners): add infra ownership layer for CI, scripts and build files by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/8573
• ci(multi-gpu): include torchrun test files in the execution summary by @nvamyt in https://github.com/flashinfer-ai/flashinfer/pull/8470
• perf(cake_dense_projection_gemm): faster K1 forward programs for small-N rows on 152-SM sm_100a and sm_103a parts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8794
• feat(attn_scores): add selective MQA TopK for prefill (indexer-throwaway topk) by @rosenrodt in https://github.com/flashinfer-ai/flashinfer/pull/7325
• docs(moe): document the gated FC1 [X1; X2] layout and clamp/offset mapping, add deinterleave_gated_rows, pin the MXFP8 clamp/offset numerics by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/8433
• perf: group tiny-KV GQA context requests on SM103 by @saltyminty in https://github.com/flashinfer-ai/flashinfer/pull/8736
• perf(cake_gdn_det_kcp): cheaper eager host path for the deterministic KCP package by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8817
• perf(cake_kda): compute warps load the first task's entering state in three new bounded direct N32 turnover builds of BF16 KDA prefill for 128-token launches (sm_103a) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8823
• ci: report failed and cancelled matrix lanes in test summary by @YaoleWu in https://github.com/flashinfer-ai/flashinfer/pull/7996
• prefill: use fa2 backend for small per-request query lengths by @zz774ever in https://github.com/flashinfer-ai/flashinfer/pull/5188
• perf(cake_dense_projection_gemm): exact-FP32 cluster SIMT router forward at 1 to 192 tokens on 152-SM sm_100a / sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8807
• fix(cake_rmsnorm): sm_100a and sm_107a dw-accuracy backward routes by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8827
• perf(cake_batch_deepgemm_fp8): per-cell sm_107a routes for N=4096 K=7168 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8824
• perf(cake_gdn_det): serial backward state scan for priced split plans on R200 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8820
• perf(cake_rmsnorm_det): FP32-fold column-task backward for small token counts on SM103 and SM107 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8813
• perf(cake_gdn_det): split the fused forward scan and output over the value halves on R200 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8805
• feat(batch_invariant): bmm_fp8(batch_invariant=True); the dispatcher becomes a ranking heuristic by @bkryu in https://github.com/flashinfer-ai/flashinfer/pull/8453
• fix(autotuner): stop tuning when a failed tactic leaves the CUDA context unusable by @arpera in https://github.com/flashinfer-ai/flashinfer/pull/8345
• fix(gemm): skip CUTLASS in gemmBF16 when N or K is not a multiple of 8 + fix TGV kernel by @arpera in https://github.com/flashinfer-ai/flashinfer/pull/8226
• fix(comm): warn on the deprecated UlyssesCommunicator max_elems alias by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/8583
• perf(cake_dsv4): SM107 four-stage FP8 64-head M64 program issues QK(t) without waiting for PV(t-1) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8828
• fix(prims_ts): persist compiled MoE GEMMs and fix the large-batch LDGSTS TensorMap abort by @bobboli in https://github.com/flashinfer-ai/flashinfer/pull/7585
• feat(gdn): Rubin (sm_107a) build of the UMMA ReplaySSM MTP u-cache flush kernel, with uGPU localization by @ameynaik-hub in https://github.com/flashinfer-ai/flashinfer/pull/6791
• perf(cake_vsa_sm120): pack the last partial wave of the SM120 VSA work plan by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8829
• docs: fix API indexes and RMSNorm RST rendering by @kangbintNV in https://github.com/flashinfer-ai/flashinfer/pull/8464
• fix(cake_gdn_det): key the chunk-table cache on the dhu_scan pin by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8834
• perf(gemm): avoid constructing a Stream in _get_stream_cache_buf by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/8696
• fix(norm): cap the RMSNorm cluster at the device's largest launchable size by @jimmyzho in https://github.com/flashinfer-ai/flashinfer/pull/8752
• perf(cake_comm): keep one WGMMA group in flight in the SM90 QKV norm-RoPE exchange by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8826
• feat(batch_invariant): bias for mm_bf16(batch_invariant=True) by @bkryu in https://github.com/flashinfer-ai/flashinfer/pull/8535
• fix(cake_mla): keep small V block scales in the NVFP4 MLA paged decode wide route (#7163) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8837
• perf(cake_hopper_msa): single-pass exact top-k select for the plain 2-chunk program by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8839
• perf(cake_rmsnorm_silu_gate_train): FP32 two-sum fold on SM100 from 800 rows per SM by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8840
• feat(mamba): add inference causal conv1d prefill and state update by @leonardHONG in https://github.com/flashinfer-ai/flashinfer/pull/8273
• feat(autotuner): more reproducible, auditable tactic selection and representative routed-MoE profiling routes by @rohithrr9 in https://github.com/flashinfer-ai/flashinfer/pull/8451
• fix(gemm): validate cached GEMV tactics while preserving warmup reuse by @righthandabacus in https://github.com/flashinfer-ai/flashinfer/pull/8727
• perf(cake_dense_projection_det_gemm): launch the K1 entry points from a binding of their key by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8843
• fix(cake_rmsnorm_train): twosum dw sums for the largest token counts of the sm_103a H4096 / H5120 backward by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8849
• perf(cake_kimi_k3_attn_res): row-capped L2 first-chunk prefetch programs for ten B200 and three B300 shapes by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8847
• perf(cake_dsa_indexer_det): an FP8 wide scan program on sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8841
• fix(prims_ts): index split-KV partials by the runtime split count by @xuanzic in https://github.com/flashinfer-ai/flashinfer/pull/8078
• fix(prims-ts): recover four MoE perf regressions from #6832 by @jimmyzho in https://github.com/flashinfer-ai/flashinfer/pull/7833
• perf(cake_moe_finalize_allreduce_fusion): shared-expert prefetch programs at world sizes 8 and 2 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/8864
• Update CODEOWNERS by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/8871
• fix(experimental/b12x): deterministic pivot-tie fill in persistent top-k2048 by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/8561
• Modify CODEOWNERS for directory ownership by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/8872
• fix(attention): preserve FA2 history when the running maximum is unchanged by @yanminglai in https://github.com/flashinfer-ai/flashinfer/pull/6908

Auto-generated notes truncated to fit the GitHub release body limit. Full changelog: v0.7.3rc1...v0.7.4rc1