flashinfer-ai/flashinfer v0.7.3

Release v0.7.3

on GitHub

What's Changed

• perf(cake_kimi_k3_latent_moe): plan from the device SM count; qualify 148- and 152-SM parts (SM100a, SM103a) by @yyihuang in #6109
• perf(cake_kimi_k3_attn_res): qualify the native ports on 148- and 152-SM parts (SM100a, SM103a) by @yyihuang in #6108
• perf(cudnn): reduce attention execution and planning overhead by @YangXu1990uiuc in #5526
• feat(cake_ssd_combined): <128-token calls, seq_idx guard, grouped epilogue loads, any chunk_size via cu_seqlens (SM100 / SM103) by @yyihuang in #6100
• perf(prefill): defer CUTLASS work counting until needed by @YangXu1990uiuc in #5747
• fix: initialize paged fallback metadata with an explicit KV maximum by @YangXu1990uiuc in #6079
• fix(ci): handle isolated SM107 nightly tests by @cindyzxq in #5673
• bump version to 0.7.2 by @kahyunnam in #6158
• perf(cake_deepgemm): folded weighted-sum epilogue for the 64-head FP8 MQA logits program; tiered sm_100a paged admission (SM100a, SM103a) by @yyihuang in #6097
• fix: package SM90 BF16 Cake MegaMoE JIT sources by @cindyzxq in #6116
• feat(mla): unify backend execution and add wrapper autotuning by @saltyminty in #6106
• feat(cake_dsa_indexer): round 2 generated programs -- host-dispatched scan pipeline levers and the staged / two-level prefix-popcount rank finalize with pools sized by max_seqlen_k, bulk-copy row I/O, a 16-slot thread form for top_k > 2048 and an SM107 persistent two-row form for top_k <= 2048, bitwise identical (SM100 / SM103 / SM107) by @yyihuang in #6056
• perf(cake_all_gather_matmul): overlap the gather with the GEMM — persistent arrival-ordered schedule, SM push for small shards, PDL chaining (SM100a, SM103a) by @yyihuang in #6153
• perf(cudnn): bound cached MLA prefill geometry by @YangXu1990uiuc in #6134
• fix(cudnn): gate backend="auto" on a caller-owned block table's declared cache length (cudnn-frontend 1.30) by @vedaanta in #6150
• perf(cake_gdn): Qwen3.5 BF16-state T=1 decode bands and five prefill head splits (SM100a, SM103a) by @yyihuang in #6131
• feat(cudnn): backend="auto" takes cuDNN for single-token d256 paged decode in the measured CTA band; CUDA-graph wrappers declare replay to cudnn-frontend 1.31+ by @vedaanta in #6152
• feat(cake_diffusion): MiniMax-H3 SM120 fused quantized full MLP, FC1 + SwiGLU + FC2 + gated residual (RTX 5090 / RTX PRO 6000) by @yyihuang in #6169
• test(cake_backend): cross-backend DeepSeek-V3 routing coverage on real-logit, tie and large-T inputs by @yyihuang in #6171
• fix(cake_all_gather_matmul): re-allocate past torch's stale NVSHMEM rendezvous handle on workspace growth (SM100a, SM103a) by @yyihuang in #6157
• fix(ci): isolate test_sharding child pytest runs from nightly PYTEST_ADDOPTSFix sharding plugin test pytest addopts by @nvamyt in #5722
• fix(rope): gate cuTile FP8 quantization on SM89+ by @cindyzxq in #6188
• feat(moe): add Rubin locality-domain execution by @soodoshll in #5276
• test: bound peak memory in mxfp4 groupwise group GEMM test (32GB OOM) by @waynehacking8 in #3594
• feat(moe): enable TRTLLM-GEN FP8 routed MoE on SM107 (Rubin) by @Vinnie6167 in #5390
• [Patch] Enable SM107 on kda-prefill-cute by @johncalesp in #6201
• feat(cake_deepgemm): SM120 paged FP8 MQA lightning-indexer logits for RTX PRO 6000 / RTX 5090 (experimental generated-program export) by @yyihuang in #6180
• perf(cake_sparse_mla): FP8 H128 prologue rendezvous + BF16 SWA128 two-chunk online softmax (SM100/SM103) by @yyihuang in #6193
• feat(moe): add the remaining Prims-TS quant pairs to the unified API by @feih-nv in #5718
• feat(cake_deepgemm): SM120 paged MQA-logits indexer: one Q atom per request for next_n 4, strided per-layer cache views by @yyihuang in #6199
• feat(cake_warp_decode): add clamped DeepSeek-V4-Flash routed NVFP4 decode for SM100 by @yyihuang in #5558
• refactor(cake_dsv4): fold the NVFP4 decode knob clones into kernel templates and share the binding host code (SM100/SM103) by @yyihuang in #6190
• perf(cake_moe_finalize_allreduce_fusion): hoist the residual row ahead of the Lamport poll and gather the routes four at a time by @yyihuang in #6166
• fix(moe): reject unsupported routing modes and per-token scales in every unified runner by @feih-nv in #5663
• feat(cake_deepgemm): SM120 paged MQA-logits indexer: next_n 3, 5 and 6 programs for 32 and 64 heads by @yyihuang in #6219
• perf(cake_dsv4): dequantize the NVFP4 decode cache with the hardware QMUL4 via PTX ISA 9.4; require CUDA 13.4 (SM100/SM103) by @yyihuang in #6215
• [Perf] Reduce cuDNN linear-attention host overhead by @Anerudhan in #6218
• feat(moe): add cuDNN grouped-GEMM backends to the unified MoE API by @Aneureka in #5453
• perf(cake_diffusion): MiniMax-H3 SM120 quantized MLP, FP8 FC1 raster group 64 and NVFP4 FC1 producer warpgroup + staged y emission (RTX 5090 / RTX PRO 6000) by @yyihuang in #6196
• feat(cake_kimi_k3_attn_res): serving-shape routing (M 1..16384) + snapshot-write variant for SM100 / SM103 by @yyihuang in #6070
• perf(cake_minimax_h3_varlen_attention): round-2/3 BF16 varlen attention schedule and segment plan cache (SM100a, SM103a) by @yyihuang in #6062
• feat(cake_dsa_indexer): round 3 generated programs -- SM100 wide-kind register split (232 math / 40 I/O per lane, host-dispatched), bitwise identical to round 2 on SM100 / SM103 / SM107; lean export (shared headers, lever table in tests, cached device queries), SASS unchanged by @yyihuang in #6230
• perf(cake_sampling): round 11 -- exact early termination of the leader radix passes, per-capability dispatch, hard floors (SM90, SM100a, SM103a, SM107a) by @yyihuang in #6143
• fix(tests): gate SM120-only MiniMax kernels on exact CC by @cindyzxq in #6228
• feat(cake_concat_mla_kv_quant_fp8): generated Cake kernels for the fused MLA K/V pack + fp8 cast by @yyihuang in #6163
• fix(msa): keep earlier blocks when the NVFP4 prefill replay rebases deeply by @yekerr in #6093
• perf(prims-ts): qk-bf16/pv-fp8 and all-fp8 dense fmha context perf bringup by @harrisonzhy in #5900
• fix(tests): count caching-allocator requests in the Cake DSv4 hardening no-allocation checks by @yyihuang in #6244
• perf(cake_ssd_combined): preprocess, exact-scan and chunk-parallel prefill improvements (SM100 / SM103) by @yyihuang in #6206
• ci: move the CUDA 13.4 CI image to cuDNN 9.26.0.51 by @dierksen in #5877
• feat(cli): let install-jit-cache-wheel --sm imply minimal mode by @dierksen in #6095
• fix(cubin_loader): log HTTP error response headers and body at DEBUG by @dierksen in #5980
• fix(jit): ignore stale cubin/jit-cache wheels at import so download-kernels can replace them by @dierksen in #5892
• ci: retry JIT cache image and dependency downloads by @dierksen in #5748
• fix(moe): route the cuDNN grouped-GEMM runners through the shared pack contract by @yyihuang in #6264
• feat(cake_all_gather_matmul): serve torch's default CUDA symmetric-memory backend and capture the prepared launcher (SM100a, SM103a) by @yyihuang in #6104
• feat(moe): add TRTLLM Gen SwiGLU StepFun support by @jiahanc in #6226
• [SM120] Add W4A8 CuTeDSL MegaMoE backend by @Euynaheh in #4632
• test: prune FP8 blockscale GEMM parameter matrix by @righthandabacus in #5423
• test: prune SM-constrained GEMM parameter matrix by @righthandabacus in #5424
• feat(moe): fuse tanh-clamped squared relu in CUTLASS epilogues by @mathemakitten in #5696
• feat: integrate cuDNN linear attention pools and bounded auto dispatch by @Anerudhan in #6259
• cudnn prefill: return a packed LSE, declared as ragged Stats by @egilliam-nv in #5161
• fix(gemm): use the SM107 CuTe-DSL kernel for mm_mxfp8(backend="cute-dsl") on Rubin by @Vinnie6167 in #6136
• test: prune GDN decode parameter matrices by @righthandabacus in #5422
• test: prune FP4 quantization parameter matrix by @righthandabacus in #5426
• feat(cake_diffusion): MiniMax-H3 full MLP block operator (BF16 / MXFP8 / NVFP4, SM100 / SM103) by @yyihuang in #6176
• perf(cake_dsa_indexer): one lowering of the weighted ReLU sum in the DSA indexer top-k programs (SM100a, SM103a, SM107a) by @yyihuang in #6195
• perf(cake_ssd_combined): round-4 exact-scan schedule, chunk-parallel host model and graph-capturable prepared submission (SM100 / SM103) by @yyihuang in #6291
• perf(cake_deepgemm): dynamic chunk scheduler for the paged FP8 MQA logits; 64-head dense programs: early Q-stage release, short-KV one-split routes and a prologue prefetch; refreshed admission tables (SM100a, SM103a) by @yyihuang in #6118
• fix(cake_all_gather_matmul): proxy fence after the readiness acquire by @yyihuang in #6254
• perf(cake_minimax_h3_varlen_attention): round-6 SM100a single P barrier and tail-round Q-split planner (SM100a, SM103a) by @yyihuang in #6258
• perf(cake_dsv4): add the H64 tile member to the NVFP4 sparse-MLA decode family (SM100/SM103) by @yyihuang in #6262
• test: prune RoPE parameter matrices by @righthandabacus in #5416
• perf(cake_kimi_k3_fp8_projection): round 8 — bit-exact dispatch-cell convergence, K=128 GEMM operand hand-off, cached launch receipts, short-row TMA-store fix (SM100a, SM103a) by @yyihuang in #6217
• feat(cake_kernel_rubin): Cake backends on Rubin (sm_107a / CC 10.7) — routing, BGMV MoE, concat-MLA-K, GDN decode/prefill, packed KDA T=1 by @yyihuang in #6238
• feat(cake_fused_moe): add the Kimi-K3 W4A8 MXFP4 SiTU TP8 long-prefill mixed192 chain (T >= 8192) to the Cake-exported backends (cake, cake_cute) by @yyihuang in #6154
• perf(cake_msa): regenerate the Blackwell MSA family after the CAKE-1183 re-baseline (SM100/SM103) by @yyihuang in #6315
• chore(cake_fused_moe): re-render the Kimi-K3 MXFP4 SiTU MoE export (cake, cake_cute) from the merged Cake main revision by @yyihuang in #6320
• fix(ci): validate nightly kernel packages by @cindyzxq in #6283
• perf(cake_ssd_combined): exact-scan epilogue ring + calibrated schedule-variant rule (SM100 / SM103) by @yyihuang in #6330
• perf(cake_deepgemm): per-architecture 64-head FP8 dense texts, static-preferred windows and two new paged geometries (SM100a, SM103a) by @yyihuang in #6318
• perf(cake_kimi_k3_fp8_projection): round 9 — cluster split-K re-association cells, K=128 / 192-wide GEMM cells, fused-shape re-tune, quant_done barrier (SM100a, SM103a) by @yyihuang in #6335
• perf(cake_diffusion): MiniMax-H3 full MLP FC2 BF16 epilogue with 4 warps, bit-identical re-export (SM100 / SM103) by @yyihuang in #6314
• feat(moe_ep): add fused Hopper MXFP4 MegaMoE with autotuning by @ruoqianguo in #4843
• feat(cake_qkv_norm_rope_exchange_overlap): fused QKV projection + QK RMSNorm + RoPE overlapped with the Ulysses head exchange (SM90 / SM100 / SM103) by @yyihuang in #6332
• perf(cake_dsa_indexer): SM103a dispatch set for the wide, paired and split scan programs of the DSA indexer top-k (SM100a programs unchanged) by @yyihuang in #6327
• perf(cake_deepgemm): single-launch 32-head FP8 / FP4 dense MQA logits routes, per-architecture 64-head programs and new paged next_n = 2 geometries; catalog schema v9 re-export by @yyihuang in #6325
• feat(mega_moe): SM90 pull-style BF16 CuTeDSL mega kernel by @inocsin in #6243
• perf(cake_gdn): SM107a FP64-free q-scale, fp32 T=1 tilev32 band, chunked-parallel prefill route and native packed-KDA tile64 variant by @yyihuang in #6323
• [feat] Add warp level split k mxfp8 gemm by @jiahanc in #6167
• feat(prims-ts): allow TMEM overlap for NVFP4 SwiGLU epilogues by @fcamaiora-nvidia in #6255
• feat(concat_mla_kv_quant_fp8): specialized kernel as a second backend, auto routing by head count by @brandonfzhang in #5667
• fix(moe_ep): prepare weights for all configured compute backends by @akaashrp in #6142
• feat(prims-ts): allow TMEM overlap for NVFP4 qkv overlap by @fcamaiora-nvidia in #6286
• Honor /bot run TEST_PATH in multi-GPU and multi-node CI jobs by @kahyunnam in #4643
• Fix KDA prefill metadata caches for inference tensors by @Anerudhan in #6372
• perf(cake_minimax_h3_varlen_attention): round-13 per-plan program dispatch -- dynamic ticket scheduling for long multi-wave plans (SM100a, SM103a) and a publish-tail cut for the SM100a static plans by @yyihuang in #6375
• fix: keep one-sided MoE combine workspace offset stable by @Anerudhan in #6376
• ci: rebalance A10G JIT test shards by compile group by @Anerudhan in #6377
• feat(moe): extend moe_utils to all supported arches and add more activation types for cuDNN grouped-GEMM MoE backends by @Aneureka in #6279
• test(ci): cover JIT cache image pull retries by @cindyzxq in #6351
• perf(sm12x): optimize fused MoE kernels by @EricChen02 in #5168
• feat(cute): Cute mega moe un-finalized output support by @IwakuraRein in #6253
• perf(prims-ts): tune the block-sparse/sage kernel for SM103 by @heyuhhh in #6014
• feat(cake_moe): BF16 and MXFP8 MoE training with recompute, clamped SwiGLU and variable sequence lengths on Blackwell and Rubin by @hzfan in #6053
• perf(cake_fused_moe): device-side GEMM2 path gate for the Kimi-K3 W4A8 MXFP4 SiTU mixed192 long-prefill chain (cake, cake_cute) by @yyihuang in #6411
• feat(cake_mega_moe): [SM120] MXFP4 x MXFP8 MegaMoE through MoEEpLayer (sm120_mxfp4_mxfp8_bf16_cake) by @GnohzZ in #6346
• test(logits_processor): size trials by effective support by @righthandabacus in #5515
• feat: uniform_q_len planner hint + specialized SM90 fp8-KV kernel for speculative-decoding target verify by @brandonfzhang in #6144
• ci: exclude sharding infrastructure tests from GPU jobs by @righthandabacus in #6431
• Introduce a local sccache MVP by @BrianXDong in #6268
• fix(rope): require SM90 for cuTile FP8 rope quantization by @kahyunnam in #6378
• perf(cake_sparse_mla): BF16 lever batch: online-softmax chunking, prologue rendezvous, split-KV reducer lane weights (SM100/SM103) by @yyihuang in #6220
• fix(comm): synchronize LL clusters before peer shared-memory stores by @benchislett in #5652
• feat(topk_varlen): windowed (prefill) mode for gvr_2: row_starts, register-family routing (max_seq_len), absolute_indices (TRT-LLM #18702 port) by @dhiraj113 in #5304
• feat(moe): support swiglu_limit in the SM107 CuTe DSL NVFP4 MoE by @Vinnie6167 in #6374
• chore: add righthandabacus to code owners by @righthandabacus in #6432
• perf(cake_kimi_k3_fp8_projection): round 10 — cluster split-K pair GEMM, prefetched stream-K, 128×384 pair-tile GEMM, interleaved confirm protocol and dispatch-table re-tune (SM100a, SM103a) by @yyihuang in #6433
• feat(cake_msa): Cake NVFP4 MSA decode use_cache_hit union programs for sm_100a and sm_103a by @yyihuang in #6387
• fix(prims-ts): compile batched GEMM under CuTe DSL 4.8 by @jimmyzho in #6137
• reduce-frost-host-overhead by @yanqinz2 in #6309
• revert(moe): re-enable PDL in the trtllm-gen fused-MoE path on SM107 (Rubin) by @jimmyzho in #6271
• fix(tests): keep cuTile groupwise-GEMM and block-sparse tests inside the CI time budget by @jimmyzho in #6110
• refactor(concat_mla_kv_quant_fp8): per-backend module registry, backend-neutral names by @brandonfzhang in #6373
• fix(cake_fmha): cake fmha smallm hang and graph launch by @AichenF in #6061
• perf(cake_all_gather_matmul): 128-wide main tile for small decode shards and a wide SM push for 8 MiB peer payloads (SM100a, SM103a) by @yyihuang in #6440
• fix(prims-ts): run the PrimTS attention suites on SM107 and fix FP8 x NVFP4 decode there by @Vinnie6167 in #6089
• [SM90] Add W4A8 MegaMoE support by @ouershua2299 in #6343
• perf(cake_deepgemm): fused 32-head FP8 routes, next_n-4 paged text, static windows (SM100a, SM103a) by @yyihuang in #6413
• fix(msa): fence generic SMEM stores before tcgen05.mma in the pinned NVFP4 decode by @jimmyzho in #6138
• perf(cake_concat_mla_kv_quant_fp8): 4 + 2 head-group plan at 12 heads (four pairs per warp, remainder last) by @yyihuang in #6441
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM103 single-token fused quant+route FC1 by @yyihuang in #6194
• perf(cake_msa): faster Blackwell MSA decode programs (FP16 flat ring, per-arch uniform-FP8 stages, FP8-KV q1 prefetch) by @yyihuang in #6397
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM103 large rows with an 8-bit per-tile FC2 partial by @yyihuang in #6248
• perf(cake_concat_mla_kv_quant_fp8): re-export the four Cake programs with bit-placement BF16 widening by @yyihuang in #6444
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM103 4096-token rows with the 8-bit per-tile FC2 partial by @yyihuang in #6329
• perf(cake_msa): faster Blackwell MSA Top-K select programs (keq fold, 32-query instance, plan refit; bit-identical indices) by @yyihuang in #6410
• fix(tests): skip native Blackwell GDN on CUDA 12 by @nvamyt in #6353
• perf(cake_deepgemm): FP4 dense MQA single-launch scan routes run the in-kernel ScheduleMeta scan before the barrier/TMEM setup (re-export of the two fused-scan programs) by @yyihuang in #6390
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM100 single-token row with the fused quantization + routing FC1 by @yyihuang in #6340
• fix(cake_dsa_indexer): racecheck-clean verdict hand-off (single-lane verdict reads), regenerated scan programs (SM100a, SM103a) by @yyihuang in #6446
• fix(cake_msa): Blackwell MSA FP8 decode programs no longer hang (two-phase mbarrier lead); M16 decode programs drop four dependency-free sync sites (bitwise-identical outputs) by @yyihuang in #6443
• feat(cake_msa_icp_indexer): Cake MiniMax-M3 ICP indexer chain programs (fused Top-16 scorer + cross-rank exchange merge, TP2/TP4) for sm_100a and sm_103a by @yyihuang in #6450
• feat(moe_ep): expose selected MegaMoE tuning knobs by @akaashrp in #6415
• fix(cake_fused_moe): ship every route layout and the odd weight-tile dense GEMM2 arm in the Kimi-K3 MXFP4 SiTU MoE export (cake, cake_cute) by @yyihuang in #6452
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM100 large rows with the 8-bit per-tile FC2 partial by @yyihuang in #6341
• perf(cake_msa): Blackwell MSA Top-K streaming select 1.39-1.41x faster (bank-conflict-free candidate dump, one barrier per argmax round); small-grid stage-only plan rule (bit-identical selections) by @yyihuang in #6459
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM100 16384-token row with per-CTA privatized routing by @yyihuang in #6442
• perf(cake_msa): faster Blackwell MSA prefill programs — union-route selection census, Q-resident long-route order, two-stage folded-GQA rings, shared-TMEM QK/PV issue order, tile interleave, host-side cu_seqlens_k removal (rounds 4–9 levers) by @yyihuang in #6463
• Fix SM120 W4A8 scalar-scale MMA on CuTe DSL 4.9 by @hongw-nv in #6417
• perf(cake_diffusion): bitwise norm/AdaLN prologue rewrite for MiniMax-H3 FC1+SwiGLU and fused MLP (SM100/SM103) by @yyihuang in #6474
• perf(cake_diffusion): MiniMax-H3 SM120 FC1+SwiGLU — FP8 mxf8f6f4 unit-scale form on GeForce, raster group 64, NVFP4 producer-warpgroup form by @yyihuang in #6475
• ci(tests): separate node, GPU, and host-heavy isolation by @righthandabacus in #6438
• perf(cake_dsv4): overlap the next tile's V transform with PV in the NVFP4 sparse-MLA decode persistent member (SM100/SM103) by @yyihuang in #6357
• fix(moe_ep): correct graph warmup and non-root test setup by @akaashrp in #6379
• feat(moe_ep): accept fixed knobs when creating MegaMoE workspaces by @akaashrp in #6416
• docs: document MegaMoE compile diagnostics flag by @kangbintNV in #6412
• docs: complete MiniMax H3 FC2 weight helper docstrings by @kangbintNV in #6396
• docs: fix live documentation audit findings by @kangbintNV in #5494
• feat(cake_gdn): FP32-state initial prefill records for the Qwen3-Next / Qwen3.5 TP2/TP4/TP8 geometries and the re-delivered Cake CP-prefill programs (sm_100a, sm_103a) by @yyihuang in #6478
• perf(trtllm-gen decode): keep CGA reduction for small-batch long-KV decode; fuse v_scale in the decode wrapper by @jimmyzho in #6267
• feat(cake_quant_lora_gemm): fused NVFP4 / FP8 GEMM with a runtime BF16 LoRA correction (mm_nvfp4_lora, mm_fp8_lora) by @yyihuang in #6324
• Add small-shape W4A16 MegaMoE tactics and model benchmarks by @zianglih in #6277
• fix(moe_ep): defer steady-state signature after dummy warmup by @nvamyt in #6349
• gemm: let mm_bf16 auto consider the CuTe DSL backend by @Vinnie6167 in #6392
• perf(moe): optimize sparse SM107 MegaMoE decode by @YangXu1990uiuc in #6447
• perf(cake_fmha): round-5 E4M3 head_dim 128 balanced DCP programs and the hd64 / hd256 uniform-batch fast path on SM100/SM103 by @yyihuang in #5913
• fix(gdn): honor intermediate cache view strides by @yyihuang in #4572
• feat(comm): add Kimi K3 H7168/K16 MNNVL CuTe DSL profiles for TP4 and TP8 by @benchislett in #5701
• fix(tests): initialize SM107 staging kernel capability stub by @YangXu1990uiuc in #6509
• feat(allreduce): Add support for Static FP8 Output Quant in CUTEDSL MNNVL AR by @benchislett in #5533
• perf(cake_alpha_moe): faster NVFP4 routed-MoE routes on sm_100a/sm_103a (self-routing up kernels, K-256 W2 panels, per-CC dispatch) by @yyihuang in #6511
• perf(cake_alpha_moe): dispatch the W8A8 fused MoE kernel by (M, K) across three pipeline variants by @yyihuang in #6512
• refactor-moe-engine-to-avoid-hardcoded-registry-table by @yanqinz2 in #6503
• perf(cake_mok): imbalance-aware comm split, earlier backward replay dispatch and an atomics-free load count (SM100a, SM103a, SM107a) by @yyihuang in #6439
• perf(cake_deepgemm): SM120 paged MQA-logits indexer: batch-aware Q-atom programs, DeepGEMM block-table cache, num_heads metadata kwarg by @yyihuang in #6530
• feat(cake_comm): MNNVL MoE all-to-all cake backend: fused quantized combine, vectorized BF16 combine, fused sanitize (regenerated sm100/sm103 bundles) by @yyihuang in #6545
• perf(cake_sparse_mla): FP8 prologue levers: h64 source-exact prologue overlap, low-head prefill index staging, h8/h16 setup order (SM100/SM103) by @yyihuang in #6550
• perf(cake_kimi_k3_fp8_projection): round 11 — static / grid-stride GEMM schedules, PDL-after-weights loader, 19 table adoptions (SM100a, SM103a) by @yyihuang in #6552
• perf(cake_mok): optimizations, B200 MXFP8 imbalance split, opt-in GB300 EP8 split (SM100a, SM103a; stacked on #6439) by @yyihuang in #6543
• perf(kda): prefer cuDNN within measured prefill latency margin by @Anerudhan in #6506
• test: use parametrize_product for topk_varlen by @righthandabacus in #6383
• test(logits_processor): share effective-support trial sizing by @righthandabacus in #6513
• Canonical-layout NVFP4 W4A4 MoE in CuTe DSL on SM12x by @stecasta in #5863
• Add SM120 Frost Moe Grouped Gemm Support for Swap-AB, MXFP8, NVFP4 and MXFP8×MXFP4 by @yihuawei in #6504
• feat(cake_dense_projection_gemm): GLM-5.2 dense projection GEMMs (grad, batched MLA, FP32 router) for SM100/SM107 by @yyihuang in #5753
• test: use parametrize_product for pod_kernels by @righthandabacus in #6384
• refactor(cake): self-contained CAKE package directories behind thin core facades (#6553 step 1) by @yyihuang in #6571
• fix(gdn): honor transposed output state pools on the SM100 CP route by @zzz0906 in #6064
• fix(cake_moe_grouped_gemm): rotate wgrad tensor-map slots so a group switch never edits a map an in-flight TMA load still reads (#6570) by @yyihuang in #6572
• Gdn/umma replayssm ucache flush by @ameynaik-hub in #6274
• fix(tests): skip invalid-ID embedding trap test under compute-sanitizer by @xinhe-nv in #6460
• perf(norm): SM107 launch config for add_rmsnorm_fp4quant by @jimmyzho in #6133
• feat(cake_gdn): host-free non-CP prefill launch -- optional cu_seqlens_cpu, no device-to-host sync, launch ABI v2 by @yyihuang in #6519
• [Feat] Support Muse-Spark-Open with fused normalization and TP/EP MoE optimizations by @jiahanc in #6544
• fix(ci): handle CuTe-DSL JIT cache misses by @cindyzxq in #6492
• fix(autotuner): coordinate cache hits before distributed profiling by @YangXu1990uiuc in #6185
• feat(topk_varlen): gvr_2 decode: fused page-table output (page_table), decode latency floor, K=2048 clustered admission, two-CTA slab for unsplit rows by @dhiraj113 in #5312
• feat(moe_ep)!: sync NVLinkOneSidedAlltoAll with TensorRT-LLM's kernels; remove the deprecated MoeAlltoAll/MnnvlMoe by @bobboli in #6047
• feat(cake_quant_lora_gemm): prepared NVFP4 LoRA adapter (SVDQuant-equivalent fold) and re-export of the fused quantized-base GEMM + runtime BF16 LoRA kernels by @yyihuang in #6581
• perf(cake_ssd_combined): fused exact-scan programs, host routing, look-back chunk-parallel, exactpre family (SM100 / SM103) by @yyihuang in #6582
• feat(cake_msa_icp_indexer): add world-2 exchange-merge geometries (8,8), (4,8), (2,8), (2,6), (1,6) for sm_100a and sm_103a by @yyihuang in #6580
• refactor(cake): build.json manifests, shared cake_build loader, header-free bindings; MoK sm_107a BF16/MXFP8 kernels (#6553 steps 2-4) by @yyihuang in #6584
• fix: give concurrent streams disjoint row_states buffers in multi-CTA top-k by @waynehacking8 in #3625
• fix(moe, trace): validate the pack contract in SM12xNvfp4Runner and register the new comm trace modules by @yyihuang in #6591
• perf(cake_dsv4): launch the NVFP4 sparse-MLA decode split-KV merge with programmatic dependent launch (SM100/SM103) by @yyihuang in #6578
• test(gdn): exercise the slot-slice fallback in fla_scatter_padded_pool by @SamMausberg in #5382
• add-new-decoding-pipeline-for-blackwell-megamoe by @yanqinz2 in #6565
• test(sampling): size frequency trials by effective support by @righthandabacus in #6514
• refactor(cake): behavior switches as keyword arguments; backend tests import only the package (#6553 asks 8, 9) by @yyihuang in #6610
• feat(cake_stepfun): Cake backend for the StepFun SwiGLU fused MoE, full path (SM100a, SM103a) by @yyihuang in #6046
• perf(cake_fused_moe): swap-AB cluster split-K GEMM1 forms, grouped dense raster and the CuTe DSL cluster rendezvous fix for the Kimi-K3 MXFP4 SiTU MoE export (cake, cake_cute) by @yyihuang in #6577
• fix(nvfp4_sparse_mla_decode): ship the JIT sources in wheels by @stu-cao in #6454
• feat(cake_mla): experimental Cake dense NVFP4 MLA decode for the NVFP4 paged latent cache (SM100a/SM103a, CUDA 13.4) by @yzh119 in #6242
• update-blackwell-magemoe-heuristics by @yanqinz2 in #6612
• perf(cake_msa): faster Blackwell MSA uniform-FP8 decode and Top-K programs — epilogue overlap and QK ahead of PV (D-U1, D-U2, D-U2b), streaming Top-K merge network (S2), tradix v2 radix layout, per-architecture topk_select (CAKE-1183 round 10) by @yyihuang in #6518
• test(cake_sampling): size frequency trials by effective support by @righthandabacus in #6515
• fix(cake_quant_lora_gemm): launch the gated SM90 FP8 LoRA program with raster group 1 by @yyihuang in #6705
• test(cake_fused_moe): regression tests for the plain chain's GEMM1 prefetch geometry helper of the Kimi-K3 MXFP4 SiTU MoE plan (cake, cake_cute) by @yyihuang in #6703
• perf(cake_gdn): faster Blackwell GDN prefill programs for 84 records (sm_100a, sm_103a) by @yyihuang in #6706
• fix(cake_mla): pass the causal key count to the FP8 page-64 MLA decode route by @yzh119 in #6629
• perf(cake_deepgemm_kgroup_gemm): n256 schedule tier for the FP32-accumulate l2 geometry on 152-SM devices, and a rotated tile order for its M = 5120 / N = 2304 / 3-group FP32-accumulate key on both devices (SM100a, SM103a) by @yyihuang in #6428
• perf(cake_deepgemm_sparse_mqa): K-gated evict_first KV gather for the paged logits programs, a single-writer metadata fix, and programmatic dependent launch of the logits stage by default on the paged layouts (SM100a, SM103a) by @yyihuang in #6429
• perf(cake_deepgemm_fp4_gemm): cooperative mbarrier initialisation on six routes, early load start on the BN224 / BN256 routes, and a cross-pass epilogue pipeline on the normal BN160 program (SM100a, SM103a; outputs bitwise unchanged) by @yyihuang in #6524
• perf(cake_deepgemm_mixed_gemm): cooperative mbarrier initialisation, early load start, and a split-K route for the 148-SM M = 16 / N = 4608 / K = 5120 swap-AB configuration (SM100a, SM103a; shipped programs bitwise unchanged, split-K output deterministic) by @yyihuang in #6525
• fix(cake_mega_moe): route a topk_idx of -1 to no expert in mega_moe_v3 by @yzh119 in #6627
• deslop(cake_mxfp4_situ_moe): fix pack_g1_geom, delete the 16 unreachable stages, share the dense GEMM sources between the PDL and _nopdl modules by @yyihuang in #6696
• deslop(cake_mok_bf16): share the fused bindings and BF16 kernels across targets, one launcher per shape, compact sources.json, drop the uncalled helper by @yyihuang in #6690
• deslop(cake_kda): render bindings, fold clone bodies, drop dead PTX host code by @yyihuang in #6712
• deslop(cake_dsv4): SM120 mechanical cleanup (steps 1-3) and prefill planner sweep by @yyihuang in #6708
• deslop(cake_sampling): delete the 42 stage-1 kernel builds the host dispatch never selects by @yyihuang in #6685
• deslop(cake_fmha): caller-LSE fix, fold the kernel clones and per-arch copies into one source per family, dedupe host code, add the fp8 decode GPU test by @yyihuang in #6704
• perf(cake_gemm): launch CAKE fp8_sm90_hint with a dispatched grid at raster group 2 by @yyihuang in #6716
• deslop(cake_gdn): remove dead text, merge clone bodies, render host shims by @yyihuang in #6707
• fix(cake_dense_projection_gemm): the auto stream-K plan never plans a unit without a K step by @yzh119 in #6625
• perf(cake_deepgemm_batched_gemm): evict-first B-operand loads on the swap-AB batched FP8 GEMM programs (SM100a, SM103a; outputs bitwise unchanged) by @yyihuang in #6526
• perf(cake_ssd_combined): capability-routed one-launch and tail-wave exact-scan programs for Blackwell by @yyihuang in #6715
• test(bgmv_moe): order the side stream behind the pointer-table writes in TestBgmvMoeStreams by @yyihuang in #6722
• deslop(cake_warp_decode): remove dead JSON, FFI exports and host code; move checks to tests by @yyihuang in #6723
• fix(cake_mla): skip empty KV splits in the FP8 page-64 MLA decode producer by @yzh119 in #6718
• deslop(cake_vsa): fold duplicated sources and remove the SM90 CuTe DSL build by @yyihuang in #6719
• perf(cake_dsv4): route 16/32/64-head FP8 prefill rows to the single-CTA M64 body by @yzh119 in #6724
• feat(mla): read DeepSeek-V4.1 inline-scale KV records natively on SM120 by @lucifer1004 in #6616
• fix(sampling): cast per-request top_k to the dtype the kernel reads it with by @JulianZJN in #5348
• chore: update Attention, GEMM, and MoE code owners by @YangXu1990uiuc in #6732
• chore(artifacts): bump trtllm-gen BMM cubins for StepFun SwiGLU by @jiahanc in #6726
• perf(cake_quant_lora_gemm): faster SM90 FP8 LoRA GEMM mainloop by @yyihuang in #6728
• feat(cake_ssd_combined): Rubin sm_107a programs for the Mamba2 SSD combined prefill by @yyihuang in #6746
• deslop(cake_dsv4): share arch-identical SM100/SM103 kernels, fold NVFP4 launchers, drop dead host paths by @yyihuang in #6731
• perf(cake_diffusion): MiniMax-H3 FC1+SwiGLU 256x448 MXFP8 tile, PDL and packed epilogue rounding; prologue scale arithmetic (SM100/SM103) by @yyihuang in #6720
• perf(cake): sm_107a (Rubin) kernels for DeepSeek fused routing, BGMV MoE and concat-MLA-K by @yyihuang in #6747
• perf(cake_quant_lora_gemm): hoisted stage wait in the SM90 FP8 LoRA GEMM by @yyihuang in #6753
• feat(cake_bf16_fp4_gemm): faster SM100/SM103 BF16 x FP4 GEMM kernels for the blackwell-native and blackwell-tiled backends by @yyihuang in #6614
• deslop(cake_deepgemm_mqa): fold SM120 and dense paged clones, share the host helper, drop unread catalog data by @yyihuang in #6733
• perf(cake_gdn): refresh Blackwell GDN prefill bodies for 84 records (guard fold, tcgen05 thread-sync fences) by @yyihuang in #6752
• [Bugfix]Pcie ipc bugfix by @qsang-nv in #6401
• perf(cake_rubin): sm_107a (Rubin) kernels for GDN prefill, GDN decode and packed KDA T=1 by @yyihuang in #6754
• deslop(cake_mamba): drop unreachable SSD sources, share the exactpre launcher, fold the STP mb4 clone by @yyihuang in #6739
• deslop(cake_kimi_k3_attn_res): shared sources, three launch bindings, tabular registry, dead planner code by @yyihuang in #6756
• deslop(cake_msa_icp_indexer): fold the scorer clones via -DBLOCK_Q, share one binding per role, drop dead host code by @yyihuang in #6748
• deslop(cake_gdn_cp): merge 22 SM107 program copies, drop unreachable h32, aliases, dead code by @yyihuang in #6749
• deslop(cake_fused_qk_rope_append): one kernel and binding per family, slim manifests, policy-gap fix by @yyihuang in #6751
• perf(cake_deepgemm_kgroup_gemm): host-built LPT tile order for seven three-group keys (M = 5120 / N = 2304 and M = 4608 / N = 5120 across bf16 / fp32 / fp32-accumulate output modes) on the N256 and BM240 schedules on SM100a and SM103a by @yyihuang in #6750
• deslop(cake_kda_bf16): render the host bindings, fold the fused-M128 clones, route kernel-less requests, trim dead host code by @yyihuang in #6758
• perf(cake_comm): fold the readiness barrier into the QKV norm+RoPE exchange producer (2 launches per call, default on SM90/SM100/SM103) by @yyihuang in #6516
• deslop(cake_minimax_h3): merge SM100/SM103 sources, share the FP8 FC1 GEMMs, fold nodelta clones by @yyihuang in #6737
• feat(cake_w8a8_megamoe_ep16): experimental W8A8 MXFP8 MegaMoE EP16 backend for SM103 by @yyihuang in #6759
• perf(attention): Enable multi-token NVFP4 KV token/head grouping by @roigarnettNvidia in #6471
• perf(prims-ts): optimize QToken-KvBlock-Sparse-Attention by @PerkzZheng in #5351
• feat(attention): FA2 per-(token, head) FP8 KV scale by @ir1ka in #5272
• deslop(cake_bgmv_moe): drop 30 never-launched kernels, fold hidden/rank/FMA clones, gate per target by @yyihuang in #6745
• deslop(cake_blackwell_bf16_fp4): drop unread MODULES fields, share validation, pin routes by @yyihuang in #6727
• perf(cake_gdn): one CTA per tile for ragged full-DV prefill batches (host launch grid) by @yyihuang in #6764
• perf(cake_comm): enable the Blackwell exchange-plan defaults for the QKV norm+RoPE exchange (SM100 / SM103) by @yyihuang in #6537
• feat(cake_rmsnorm): re-export the BF16 RMSNorm grad for sm_100a, sm_103a and sm_107a by @yyihuang in #6765
• deslop(cake_blackwell_msa): fold uniform-FP8 arch pair, share NVFP4 helpers, name arg plans once by @yyihuang in #6766
• perf(cake_all_gather_matmul): SM103a SM push route band and cta_group::2 pair main kernel for measured regions by @yyihuang in #6501
• [NVBUG-6791733][fix] Guard ragged Q tensor maps on Blackwell by @yuxianq in #6482
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM100 8192-token row: fused quantization + reset and 512-pair routing CTAs by @yyihuang in #6622
• perf(cake_fused_moe): Kimi-K3 NVFP4 SiTU experts SM100 8- and 16-token rows with evict-first weight-stream loads by @yyihuang in #6628
• feat(cake_comm): NVFP4-transport tensor-parallel all-reduce (TP2 one-shot, TP4 two-shot) with a fused residual + RMSNorm epilogue (SM100 / SM103 / SM107) by @yyihuang in #6500
• test: cover RMSNorm caller-owned output contracts by @0z5a in #4994
• test(sampling): add seeded correctness fuzz coverage by @JulianZJN in #5341
• feat(cake_comm): QkvNormRopeExchangeOverlap per-world-size producer variants and plan defaults on sm_100a / sm_103a / sm_90a by @yyihuang in #6613
• perf(cake_deepgemm_mega_gate): deterministic small-M route on a cta_group::2 split-1 template -- one exported template for M <= 16 deterministic calls (SM100a, SM103a) by @yyihuang in #6427
• perf(cake_quant_lora_gemm): W-resident strip raster for the SM90 FP8 LoRA GEMM hint program by @yyihuang in #6769
• perf(cake_ssd_combined): SM107 varlen exact-scan geometry through the aux stage and full-grid tail-pair routing by @yyihuang in #6775
• perf(cake_all_gather_matmul): SM103a CTA-pair main kernel for two measured gate/up shapes (region table with column ranges) by @yyihuang in #6566
• test: use parametrize_product for block_sparse_cutile by @righthandabacus in #6380
• feat(cake_comm): QkvNormRopeExchangeOverlap adds the SM107 (Rubin) producer programs by @yyihuang in #6620
• perf(cake_deepgemm_mega_mhc): two-lane Sinkhorn normalization chain on the 40- and 27-split mHC programs (SM100a, SM103a) by @yyihuang in #6202
• deslop(cake_kimi_k3_fp8_projection): share kernel/launcher sources, slim MODULES, drop dead host code by @yyihuang in #6780
• feat(cake_kda): packed KDA chunked training kernels for GLM-5.3-Flash with grad by @yyihuang in #6787
• feat(cake_fp8_rowwise_gemm_allreduce): fused row-wise FP8 GEMM + all-reduce backend (SM100, four ranks) by @yyihuang in #6583
• perf(moe): retune the routingCustom split-topK dispatch rule by @Vinnie6167 in #6555
• perf(cake_comm): QkvNormRopeExchangeOverlap enables words_capacity_small_m=4096 on SM90 by default by @yyihuang in #6717
• feat(cake_causal_conv1d): packed causal conv + SiLU kernels for sm_100a, sm_103a and sm_107a with grad by @yyihuang in #6782
• perf(cake_all_gather_matmul): SM100a CTA-pair main kernel for nine measured shapes by @yyihuang in #6586
• perf(moe): split the topK reduction in small-batch trtllm-gen MoE finalize by @Vinnie6167 in #6546
• perf(gemm): SM107 FP8/FP4 GEMM selection: mm_fp8 cuBLAS route, bmm_fp8 runner order, mm_fp4 split-K picks by @jimmyzho in #6521
• fix(topk): order the cached-count reset after the previous round's read in topk_clusters_exact by @StableFluffy in #6400
• feat(cake_w8a8_megamoe_ep16): regenerate the six device kernels from their Cake sources by @yyihuang in #6796
• CuteDSL MegaMoe Optimization for DSV41 by @Victor49152 in #6499
• perf(cake_all_gather_matmul): SM103a 16-CTA push and ws2 CTA-pair points by @yyihuang in #6606
• perf(gdn): Rubin (SM107) chunked-prefill kernel subclass and SM107 fp32 decode launch config by @jimmyzho in #6507
• fix(trace): derive mm_bf16_fp4 cute-dsl trace gates from the library predicate by @kahyunnam in #6270
• feat: add CuTe-DSL backend for layernorm_quant by @elwhyjay in #4335
• fix(jit): reuse artifacts downloaded by concurrent workers by @pkuwangh in #6461
• feat(gemm): SM107 CuTe-DSL blockwise FP8 GEMM backend for gemm_fp8_nt_groupwise by @jimmyzho in #6508
• perf(cake_all_gather_matmul): SM103a spill-split pair twins for gate/up by @yyihuang in #6624
• feat(cake_rmsnorm): add H1536 / H4096 routes with grad (SM100 / SM103 / SM107) by @yyihuang in #6792
• deslop(cake_dsa_indexer): fold kernel twins, share launcher bindings, drop dead registry glue by @yyihuang in #6802
• perf(cute_dsl_mla): let small CuTe-DSL MLA decode launches use more KV splits by @jimmyzho in #6710
• perf(moe): advertise FP4 tile 192 on Rubin and widen the trtllm-gen MoE tuner window by one tile below by @Vinnie6167 in #6788
• fix(tests): resolve knob cache modules from the installed package by @cindyzxq in #6472
• fix(cake_ssd_combined): regenerate the device sources with tcgen05 thread-sync fences on every thread synchronization by @yyihuang in #6808
• perf(cake_comm): direct, unrolled normalization pass for the TP2 fused all-reduce + RMSNorm with BF16 output (SM100 / SM103 / SM107) by @yyihuang in #6809
• perf(cake_all_gather_matmul): SM103a K-slice readiness, 512-row TP8 push by @yyihuang in #6742
• fix: Stage the sharding pytest plugin inside the isolated nightly directory without exposing the source checkout on PYTH by @cindyzxq in #6467
• perf(cake_msa): persistent heavy-first union prefill, streaming Top-K merge/scan (rounds 11-15 re-export) by @yyihuang in #6725
• perf(gemm): let the untuned SM107 mm_mxfp8 default pick B-reuse tactics by @qiching in #6551
• deslop(cake_nvfp4_per_token): fold kernel and binding twins, slim the registry, ship csrc in wheels by @yyihuang in #6813
• feat(cake_stepfun): StepFun SwiGLU fused MoE Cake backend in the cake-build form (SM100a, SM103a) by @yyihuang in #6801
• perf(cake_all_gather_matmul): SM103a 16-CTA push launch, 1024-row TP8 by @yyihuang in #6761
• perf(cake_causal_conv1d): per-warp staged with grad with eight channels per thread on sm_100a, sm_103a and sm_107a by @yyihuang in #6816
• perf: [PrimsTS] add BF16 N192 MoE kernel; batchM shuffled weight; fix TMA routing perf regression by @rosenrodt in #6336
• deslop(cake_sage_block_sparse_attention): one SM120 kernel source, drop degenerate modules and dead device code by @yyihuang in #6827
• feat(prims-ts): cache attention kernels and enable FP8-to-BF16 decode by @PerkzZheng in #6529
• feat(prims-ts): add native two-source sparse MLA by @PerkzZheng in #5434
• fix(cake_alpha_moe): one TMEM address per warp for the NVFP4 epilogue tcgen05.ld by @yzh119 in #6767
• fix(cake_dense_projection_gemm): every CTA-pair program drains the multicast stage releases before it exits by @yzh119 in #6763
• fix(cake_minimax_h3): wait for the peer CTA's last stage releases before the pre-attention QKV GEMM load warp exits by @yzh119 in #6820
• fix(cake_dsa_train): each warp's tcgen05.ld addresses its own TMEM lane quarter by @yzh119 in #6778
• deslop(cake_kimi_k3_vision_tower): one launcher for the 76 host bindings, fold SM100 attention copies, compact registry by @yyihuang in #6830
• perf(cake_comm): QkvNormRopeExchangeOverlap SM107 four-rank plans use the bulk-row-store build by @yyihuang in #6831
• perf(moe_ep): fuse Rubin MegaMoE forward preparation by @akaashrp in #6398
• fix(cake_mla): address each warp's own TMEM lane quarter in the FP8 page-32 Blackwell MLA decode by @yzh119 in #6828
• fix(cake_minimax_h3): wait for the peer CTA's last stage releases before the FC1 / MLP / out-proj GEMM load warps exit by @yzh119 in #6822
• fix(cake_kimi_k3_latent_moe): wait for the peer CTA's last stage releases before the tail GEMM load warp exits by @yzh119 in #6821
• fix(cake_deepgemm_mixed_gemm): wait for the peer CTA's last stage releases before the load warp exits by @yzh119 in #6819
• fix(cake_selective_state_update): sync each consumer warp before its elected stage release by @yzh119 in #6806
• fix(cake_kimi_k3_attn_res): drain tcgen05.ld with tcgen05.wait::ld before the TMEM dealloc (native programs) by @yzh119 in #6744
• perf(cake_causal_conv1d): 2-D TMA stage loads in the sm_100a/sm_103a with grad by @yyihuang in #6844
• feat(cake_gemm_allreduce): fused tensor-parallel GEMM + MNNVL Lamport all-reduce for Blackwell (SM100 / SM103) by @cowanmeg in #6493
• fix(cake_kimi_k3_attn_res): drain tcgen05.ld with tcgen05.wait::ld before the TMEM dealloc (persistent programs) by @yzh119 in #6740
• perf(cake_deepgemm_dense_mqa): v11 re-export on the folded catalog, withdrawn paged points by @yyihuang in #6579
• perf(cake_all_gather_matmul): SM103a peer-outer tile order, TP4 gate/up by @yyihuang in #6841
• fix(moe): wait for Rubin finalize reductions before shared-buffer reuse by @akaashrp in #6307
• fix(moe_ep): skip non-local routes in split expert compute by @Anerudhan in #6840
• fix(gdn): fence TMEM loads before cg1 shared_acc releases by @dundysm in #5335
• perf(cake_ulysses): interleaved-stripe NVLink all-to-all kernels for sm_100a/sm_103a by @yyihuang in #6864
• feat(cake_dsa): native NoPE sparse MLA kernels (GLM-5.3-Flash) for the 64-head DSA backend on SM100 / SM103 / SM107 with grad (#6489) by @yyihuang in #6836
• refactor(cake_fused_qk_rope_fp8_append): regenerate the kernel and route the host through the in-tree program loader by @yyihuang in #6867
• perf(cake_causal_conv1d): a shape-selected big-channel backward program (C >= 512 on sm_100a/sm_103a, C >= 1024 on sm_107a) by @yyihuang in #6874
• perf(cake_fmha): uniform-gated per-item L2 stream hints for the balanced q1 decode kernels (SM100a, SM103a) by @yyihuang in #6111
• perf(cake_rmsnorm): faster H1536 / H4096 grad (SM100 / SM103 / SM107) by @yyihuang in #6842
• fix(benchmark): use HND layout for cuDNN paged prefill by @lunarz-dev in #6846
• fix(benchmarks): skip unsupported CUTLASS MLA head counts by @yufeiwu-nv in #6451
• refactor(cake_minimax_h3_qkv_quantize_pack): regenerate the kernels and route the host through the in-tree program loader by @yyihuang in #6869
• perf(cake_w8a8_megamoe_ep16): evict-first L2 hint on the FC2 weight loads by @yyihuang in #6857
• deslop(cake_stepfun_moe): derived launchers, compact registry, routing-twin fold, 47 shared bindings, preamble hoist by @yyihuang in #6871
• deslop(cake_fp8_gemm): table-driven batch routing, shared headers, mxfp8 and grouped source folds, dead host text by @yyihuang in #6862
• deslop(cake_kda_tf32): dead host code, one device-helper header, 24 kernel clones folded onto 9 bodies by @yyihuang in #6876
• feat(gdn): prefix-cache state materialization kernel by @ameynaik-hub in #5073
• deslop(cake_kimi_k3_situ): dead selectors, one device-helper header, the FC1 pair folded, nine host bindings shared by @yyihuang in #6879
• fix(gdn): SM90 prefill wrote final states past a compact scratch for … by @omera-nv in #6849
• fix(moe): adapt localized execution to the updated PyTorch stream API by @soodoshll in #6488
• [BugFix #5139] Accept the TC-decode ultra FC2 tile in the SM12x W4A16 force_tile_config re-pin by @Shreya-gaur in #6292
• perf(cake_fmha): plan-free static one-wave program for the E4M3 head_dim-128 balanced DCP decode family (SM100a, SM103a) by @yyihuang in #6119
• fix(cake_mla): read layout-B scores from the warp's TMEM lane quarter by @yzh119 in #6590
• fix(cake_ssd_combined): cap the look-back vcount at the device's SM count by @yyihuang in #6884
• perf(cake_nvfp4_svdquant): converge the Cake NVFP4 SVDQuant GEMM backend and ship it as a vendorable package by @yyihuang in #6878
• fix(cake_minimax_h3): order the SM120 dense attention epilogue before the next Q TMA refill and close the ping-pong barrier by @yzh119 in #6847
• refactor(cake_grouped_mxfp8_quantize): regenerate the kernels and route the host through the in-tree program loader by @yyihuang in #6868
• feat(moe): Trtllm gen workspace size by @IwakuraRein in #6825
• perf(cake_sage): faster SM100/SM103 Sage-FP8 block-sparse attention and quantizers by @yyihuang in #6872
• ci: skip-all detection, cross-lane coverage, and GDN/KDA trace reference coverage (IKL-569) by @alex-ameen in #6256
• perf(cake_fmha): whole-tile static one-wave program for the BF16 head_dim-128 balanced DCP decode family (SM100a, SM103a) by @yyihuang in #6122
• deslop(cake_megamoe): drop the SM120 device-part byte copy and the uncalled EP16 host helper by @yyihuang in #6898
• perf(cake_fused_qk_rope_append): converge the BF16 fused QK RMSNorm + NeoX RoPE + paged KV append kernel and ship it as a vendorable package by @yyihuang in #6887
• fix(comm): square in fp32 in MNNVL one-shot fused allreduce+RMSNorm by @Raphael-Jin in #6818
• perf(cake_fused_qk_rope_fp8_append): converge the FP8 fused QK RMSNorm + NeoX RoPE + quantize + paged KV append kernel and ship it as a vendorable package by @yyihuang in #6888
• feat(moe): add PowLU activation to the CUTLASS fused-MoE backend by @yuan-luo in #5743
• [Experimental] Qwen 3.5 gated attention block (FROST, cuDNN Frontend) for Rubin by @RomanAnders90 in #6573
• docs(cake_fused_qk_rope_append): document the BF16 and FP8 fused QK RoPE append APIs; drop the stale pre-commit excludes by @yyihuang in #6905
• refactor(cake_concat_mla_kv_quant_fp8): re-export the MLA K/V FP8 pack through the in-tree program loader, one self-contained unit pair per head group by @yyihuang in #6901
• refactor(cake_nvfp4_attention): ship the experimental NVFP4 attention as a self-contained vendorable package by @yyihuang in #6902
• refactor(moe_ep)!: move the MoE all-to-all ops out of flashinfer.comm into their moe_ep backends; retire backend="trtllm" and the moe_comm bench; SM107 one-sided build fix by @bobboli in #6621
• perf(cake_vsa_sm90): self-contained backend package with faster small and persistent kernels by @yyihuang in #6877
• ci: report communication test completion for safe cache publication by @righthandabacus in #6815
• perf(cake_comm): in-flight row-scale read for the TP2 fused all-reduce + RMSNorm with BF16 output, shared-memory pass-2 source for TP4 two-shot (SM100 / SM103 / SM107) by @yyihuang in #6875
• perf(cake_all_gather_matmul): SM100a ws2 CTA-pair points and push row by @yyihuang in #6865
• perf(cake_msa): speed up Blackwell MSA kernels, add K-major prefill by @yyihuang in #6873
• deslop(cake_deepgemm): share the host header, fold kgroup bindings and kernel bodies, hoist the TMA encoders by @yyihuang in #6920
• perf(cake_kimi_k3_attn_res): round-2 Kimi-K3 AttnRes routes and programs for SM100 / SM103 as a vendorable cake-build package by @yyihuang in #6904
• fix(moe): make the trtllm-gen BMM default config independent of manifest order by @jiahanc in #6845
• perf(trtllm-gen): opt-in multi-wave KV split for decode when one wave gives <= 2 splits by @sungsooha in #6331
• feat(cake_mhc): four-stream mHC kernels (GLM-5.3-Flash, hidden size 4096) on SM100 / SM103 / SM107 with grad (#6486) by @yyihuang in #6897
• fix(gemm): run mxfp8 warp split-K first only when its output tiles fit one wave by @jiahanc in #6848
• perf(gemm): fuse the a_scale * b_scale product into the CuTe-DSL bmm_fp8 epilogue by @jimmyzho in #6800
• [SM90] MegaMoE: fix workspace cleanup; enable push dispatch on H200; support other models by @ouershua2299 in #6517
• feat(page): add 4over6 block scales to the NVFP4 slot-mapping KV writer by @lesj0610 in #6013
• feat(cake_gdp): add Gated DeltaProduct decode backend (sm_100a, sm_103a) by @yyihuang in #6883
• perf(cake_causal_conv1d): with grad add channel-selected backward programs for C=512 (sm_103a, sm_107a) and C >= 2048 (sm_107a) by @yyihuang in #6930
• refactor(comm): choose MNNVL handle types collectively and confirm them by allocating by @bobboli in #6734
• feat(cake_msa): Cake Hopper (SM90) programs behind the four MSA operations by @yyihuang in #6252
• perf(cake_stepfun_moe): pair-raster BF16 FC1 and two-CTA per-token NVFP4 FC2 for the StepFun fused MoE (sm_100a, sm_103a) by @yyihuang in #6934
• feat(cake_rubin_mla_nvfp4_decode): NVFP4 MLA paged decode for Rubin (sm_107a) from cake by @yyihuang in #6931
• deslop(cake_dense_projection_gemm): fold bindings and clone kernels, drop dead arms, derive test template set by @yyihuang in #6938
• refactor(cake_deepgemm_fp8_gemm): ship the DeepGEMM-derived FP8 1D1D GEMM as a self-contained vendorable package by @yyihuang in #6927
• refactor(cake_deepgemm_batched_gemm): ship the DeepGEMM-derived batched projection GEMM as a self-contained vendorable package by @yyihuang in #6928
• perf(page): stop syncing the host on every NVFP4 slot-mapping append by @lesj0610 in #6204
• feat(quantization): add CUDA backend for masked MXFP8 grouped quantization, optionally fused with SiLU-and-mul by @Aneureka in #6437
• ci: run targeted A10G scopes on the bare-run shard scripts by @dierksen in #6953
• perf(cake_ssd_combined): faster sm_107a look-back chunk-parallel SSD programs by @yyihuang in #6943
• perf(cake_sage): stage softmax metadata one step ahead on SM100/SM103 by @yyihuang in #6951
• deslop(cake_small_gemms): fold router and BMM clones, share quantized-base LoRA bindings, hoist PROGRAMS by @yyihuang in #6946
• feat(moe): add unified Frost low-latency MoE backend by @YangXu1990uiuc in #6626
• deslop(cake_minimax_h3): dedupe the varlen helpers, drop stubs and unused macros, share the norm+AdaLN kernels by @yyihuang in #6947
• perf(cake_vsa_sm90): faster persistent kernel and a compact static program by @yyihuang in #6949
• perf(cake_rmsnorm): faster H1536 / H4096 grad by row-count window and SM107 H1536 fwd (SM100 / SM103 / SM107) by @yyihuang in #6939
• feat: integrate TRTLLM-Gen sliding-window FMHA by @achartier in #4905
• perf(cake_xqa): faster SM110 XQA tree and decode routes as a vendorable cake-build package by @yyihuang in #6945
• feat(cake_minimax_h3): opt-in NVFP4-QK / FP8-PV packed-varlen attention for MiniMax-H3 SM120 by @yyihuang in #6936
• perf(cake_comm): per-architecture programs and a permuted pass-1 sum of squares for the NVFP4-transport all-reduce + RMSNorm (SM100 / SM103 / SM107) by @yyihuang in #6954
• fix(cake_mega_moe): [SM120] MegaMoE register spill with CUDA 13.3 headers; compile-check every EP by @GnohzZ in #6770
• refactor(cake_nvfp4_sparse_mla_decode): ship the NVFP4 sparse MLA decode programs as a self-contained package by @yyihuang in #6933
• feat(comm): MNNVL CuTe DSL fused BT all-reduce, dynamic FP8 output, and the mnnvl-cutedsl backend by @Vinnie6167 in #6603
• feat(cake_minimax_h3): E4M3 destination-major output for the MiniMax-H3 NVFP4 pre-attention (SM100 / SM103) by @yyihuang in #6956
• deslop(cake_kimi_k3_moe): fold clone and per-arch kernel sources, share one launcher per launch shape by @yyihuang in #6960
• feat(qsa_nvfp4): add NVFP4 QSA sparse paged attention by @lesj0610 in #6804
• fix(msa_ops): let a host tensor reach the out= refusal of msa_sparse_decode_attention by @yyihuang in #6962
• perf(cake_comm): QkvNormRopeExchangeOverlap SM90 eight-rank plans use one readiness word per destination by @yyihuang in #6964
• chore(cake_gdp): tidy module docstrings and test names by @yyihuang in #6965
• perf(cake_gqa_decode): faster SM110 GQA decode routes by @yyihuang in #6944
• feat(cake_minimax_h3): FP8 packed-varlen attention with explicit descales for MiniMax-H3 (SM100 / SM103) by @yyihuang in #6966
• perf(cake_vsa_sm90): tagged static program with single-reader K/V L2 hints and a shorter k1 prologue by @yyihuang in #6967
• feat(gemm): add CUTLASS NVFP4 SVDQuant support for SM120 by @georgeliu95 in #5228
• feat(moe): thread the explicit NVFP4 4over6 recipe into TRT-LLM gen and CUTLASS fused MoE by @feih-nv in #6403
• refactor(cake_concat_mla_k): re-export the MLA K concatenation through the in-tree program loader, one self-contained unit pair per program by @yyihuang in #6963
• feat(cake_gdp): Gated DeltaProduct decode on SM90 (sm_90a programs) by @yyihuang in #6942
• feat(moe_ep): per-token FC1 activation scale for the SM100 CuTeDSL NVFP4 MegaMoE by @qiangf-nv in #5487
• feat(prims-ts): Port sage attention to the context kernel by @heyuhhh in #6617
• deslop(cake_mega_gate, cake_mega_mhc, cake_mega_moe_v3): share bindings and arg plans, fold 40-split kernels by @yyihuang in #6973
• deslop(cake_fp8_gemm): carry the generator banner on the exporter-written files by @yyihuang in #6961
• deslop(cake_kimi_k3_situ): carry the generator banner on the exporter-written files by @yyihuang in #6957
• deslop(cake_kda_tf32): carry the generator banner on the exporter-written files by @yyihuang in #6958
• deslop(cake_trtllm_mla_blackwell, cake_kimi_k3_mla): share bindings, fold kernels, drop dead selector by @yyihuang in #6974
• perf(cake_all_gather_matmul): SM100a spill-split pair twins for the 512-row bf16 K-major gate-up calls by @yyihuang in #6975
• perf(cake_vsa_sm90): shorter prologues for the k3 and k4 small-selection programs by @yyihuang in #6976
• perf(cake_kimi_k3_attn_res): round-3 Kimi-K3 AttnRes programs and routes for SM100 / SM103 as a vendorable cake-build package by @yyihuang in #6980
• perf(cake_deepgemm_dense_mqa): v12 re-export with FP4 L2 prefetch and 64-head tile-schedule routes by @yyihuang in #6926
• perf(cake_xqa): faster SM110 XQA D512 tree routes for FP16 KV caches by @yyihuang in #6981
• perf(cake_gqa_decode): faster SM110 GQA decode short route by @yyihuang in #6982
• feat(prims-ts): support D192 sparse and sliding-window attention by @PerkzZheng in #6774
• test(moe): list cuDNN Frost in the BF16 layer-API replay test backends by @jimmyzho in #6829
• feat(cake_gdp): Gated DeltaProduct chunked varlen prefill on SM90 (sm_90a programs) by @yyihuang in #6983
• perf(cake_kimi_k3_attn_res): native K = 6 program for the (M 192, K 6) cell on SM100 at 148 SMs by @yyihuang in #6984
• perf(gemm): register the small-M MXFP8 kernels on SM107 by @jimmyzho in #6709
• perf(cake_nvfp4_svdquant): re-tuned kernels with per-architecture dispatch by @yyihuang in #6986
• perf(cake_rmsnorm): faster H512 / H2048 / H6144 grad by bulk-copy backward windows (SM100 / SM103) by @yyihuang in #6997
• fix(ci): match tests/moe_ep by path component before installing the EP stack by @dierksen in #6497
• test(sampling): size seed and offset checks by effective support by @righthandabacus in #6972
• ci: persist content-keyed JIT caches across GPU test jobs by @dierksen in #6811
• feat(prims-ts): VC-Attention BF16 recipe for fmha context by @harrisonzhy in #6296
• feat(mla): add Rubin MTP backend to the planned wrapper by @saltyminty in #6177
• fix(moe_ep): fix MegaMoE deferred routing weights and workspace profiles by @akaashrp in #6414
• refactor(topk): remove unused device function RadixSelectOneRound by @jiangyinzuo in #3037
• docs(cake_gdp): clarify the prefill backend launch and host-sync notes by @yyihuang in #6998
• perf(attention): e4m3-query FA2 tensor-core decode on SM120 by @lesj0610 in #6574
• feat(cake_kda): context-parallel packed KDA grad with precise cross-rank state passing and per-document zigzag for GLM-5.3-Flash with grad by @yyihuang in #7002
• deslop(cake_device_helpers): resolve three families' helpers from the repository headers and drop three dead ones by @yyihuang in #7017
• deslop(cake_lm_head_loss): share launch shims, fold clone kernel sources, drop dead host code by @yyihuang in #7007
• perf(cake_rubin_mla_nvfp4_decode): faster NVFP4 MLA paged decode for Rubin (sm_107a) by @yyihuang in #7004
• perf(cake_kimi_k3_attn_res): depth-2 K = 4 snapshot-write (SM100/SM103) and held nc3 K = 5 (SM103) programs by @yyihuang in #7014
• deslop(cake_mla_varq_dcp_decode): fold the kernel and binding clones, drop dead preprocessor arms by @yyihuang in #7022
• Gdn rubin stp t1 by @ameynaik-hub in #6604
• perf(cake_stepfun_moe): add tile-128 MXFP8 FC1 and FC2 kernels for the StepFun fused MoE (sm_100a, sm_103a) by @yyihuang in #7037
• fix(sampling): reserve RNG offsets for the current call by @Lonnyyy123 in #5651
• chore: make Anerudhan a code owner for all paths by @YangXu1990uiuc in #7055
• feat(cake_gdn_cp): dedicated SM 107 T-precompute program for the context-parallel GDN prefill route by @yyihuang in #7018
• perf(cake_kda): faster packed KDA chunked grad on SM100 / SM103 / SM107 by @yyihuang in #7023
• deslop(cake_moe_alltoall, cake_trtllm_moe_allreduce_union, cake_fused_norm_combine, cake_ulysses_backend): one source per clone group, one binding, one prelude header by @yyihuang in #7054
• fix(autotuner): unify cache replacement and rank agreement by @YangXu1990uiuc in #6894
• deslop(cake_mega_gate, cake_mega_mhc, cake_mega_moe_v3): carry the generator banner on the exporter-written files by @yyihuang in #7058
• feat(cake_deepgemm_mega_mhc): hidden-4096 BF16 fused mHC inference program for GLM-5.3-Flash on SM107 by @yyihuang in #7024
• perf(cake_mok): fused-kernel issue-path and TMEM hand-off optimizations for the BF16 and MXFP8 routed experts by @yyihuang in #7059
• feat(cake_fmha): aligned hd128 q1 paged decode programs for sm_100a / sm_103a by @yyihuang in #7060
• feat(attention): attention sinks over an NVFP4 KV cache in prefill and decode by @lesj0610 in #6021
• perf(gemm): re-gate BF16 CuTe-DSL direct GEMM and prune warp / cluster split-K tactics by @jiahanc in #6833
• perf(cake_deepgemm_dense_mqa): rebind 32-head FP8 routes, extend 64-head KV-split tier to Q<=1536 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7048
• perf(cake_minimax_h3): faster sm_100a program for FP8 varlen attention by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7061
• fix: retain ragged auto prefix buffers from the first plan by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7065
• fix(cake_causal_conv1d): the backward no longer stalls behind kernels resident on other streams by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7072
• fix(cake_dsa): with grad, keep nothing alive after a checkpointed forward; dkv_acc may be deferred by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7076
• deslop(cake_deepgemm_kgroup_gemm, cake_deepgemm_fp4_gemm, cake_deepgemm_mixed_gemm): carry the generator banner on the exporter-written files by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7071
• fix(cake_dense_projection_gemm): share one workspace arena per device and stream across the convenience cache by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7066
• feat(cake_qsa_nvfp4): Cake QSA NVFP4 sparse paged decode attention for SM80 and SM90 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7056
• fix(cake_mhc): keep nothing alive after a checkpointed forward (#7039) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7068
• feat(moe): extend Frost low-latency to NVFP4 and typed MX activations by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7053
• perf(cake_comm): per-CTA call state for the NVFP4-transport two-shot and fused all-reduce + RMSNorm programs, row-polled pass-1 barrier and early-launch forms (SM100 / SM103 / SM107) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7078
• feat(cake_gdp): one-wave T = 1 decode program and faster SM100 / SM103 decode kernels by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7080
• deslop(cake_lm_head_loss, experimental): import the backend on first use, add the containment test by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7067
• deslop(cake_all_gather_matmul, cake_qkv_norm_rope_exchange_overlap): one host source per sequence pair and per barrier, dead host code removed by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7079
• deslop(cake_balanced_gqa_decode): share the K/V TMA encoders, move test-only planners, derive kwargs from arg plans by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7081
• deslop(cake_moe_grouped_gemm, experimental): import the core helpers behind an ImportError guard, add the standalone-import test by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7077
• perf(moe_ep): add folded K128/N64 tuning candidate for Hopper MXFP4 x FP8 MegaMoE by @yilin-void in https://github.com/flashinfer-ai/flashinfer/pull/6465
• feat(attention): add SM12x FP8 prefill attention support for GLM-4.7 and GLM-4.7-Flash by @yilin-void in https://github.com/flashinfer-ai/flashinfer/pull/6466
• fix(cake_kda): keep no workspace alive across activation checkpointing, skip the arena zero fill and add a lower-memory arena option, with grad by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7069
• perf(cake_gdp): interleave a warp's two state rows in the T = 1 split-V decode kernel on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7086
• perf(cake_nvfp4_svdquant): tail-split register-bias programs on sm_100a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7091
• feat(autotuner): support delegated tuning protocols by @akaashrp in https://github.com/flashinfer-ai/flashinfer/pull/6534
• feat(cake_causal_conv1d): bf16 taps, bias and fp32 SiLU-gradient routes; channel tiles for C 1280/1536/2560; with grad by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7089
• perf(cake_dsa): hole-free index rows, in-place strided dout, one-launch varlen index preparation and programmatic dependent launch, with grad by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7098
• perf(cake_kda): faster context-parallel host path, wave-fill plan rule and capped graph replay, with grad- #7089 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7097
• feat(cake_vsa_sm90): add Sage-FP8 variant of the variable block-sparse attention forward by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7092
• perf(cake_all_gather_matmul): SM103a CTA-pair twins for the 512-row K-major N=2560 calls at world size 4 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7083
• perf(MegaMOE): Add support for persistent TopK kernel and explicitly skipping CuTe-DSL finalize kernel by @djns99 in https://github.com/flashinfer-ai/flashinfer/pull/5666
• fix(moe): fix trtllm moe workspace query; expose tile size by @IwakuraRein in https://github.com/flashinfer-ai/flashinfer/pull/7025
• comm: MXFP8 output for the allreduce + residual + RMSNorm fusion by @emascarenhas-nv in https://github.com/flashinfer-ai/flashinfer/pull/7031
• feat(cake_kda): document-group batching for the lower-memory arena option; with grad by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7102
• perf(cake_qsa_nvfp4): load K first, restore evict_first, 1-prmt dequant by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7103
• feat(cake_dense_projection_gemm): sm_103a programs, small-N shape rules, FP32 router backward by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7108
• perf(cake_kda): reuse the forward's carry plane in the context-parallel saved-context backward by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7117
• feat(cake_muse_spark_norm): add cake_muse_spark_norm_backend for three gated RMSNorm ops by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7119
• perf(cake_deepgemm_mega_mhc): evict-first residual stream and atomic tail flag for the hidden-4096 BF16 program on SM107 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7113
• feat(cake_tp_moe_finalize_rmsnorm): add optimized backend by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7120
• deslop(sm110): fold the shared device, binding and KV-cache blocks into headers; drop dead loader glue by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7116
• deslop(cake_blackwell_softmax): drop 3 unreachable kernels, fold clones by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7122
• deslop(cake_moe_grouped_gemm): fold the fp32 clones onto bf16, share the preamble and binding blocks, drop dead glue by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7124
• perf(cake_dsa_indexer): far-sample first threshold on the descending programs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7125
• perf: reduce Frost low-latency MoE host submission overhead by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7127
• feat(cake_deepgemm_mega_mhc): FP8 group-128 and NVFP4 output variants of the hidden-4096 fused mHC step on SM107 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7114
• feat(cake_dense_projection_gemm): fence the split-K collector's bulk read and refresh the generated programs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7129
• perf: reuse unchanged cuDNN prefill plan descriptors by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7130
• perf(cake_gdp): two-head T = 1 decode program for the many-wave rows and a reduce-scatter split-V row reduction on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7128
• fix(autotuner): rank BF16 GEMM tactics on a cold launch chain by @zcnrex in https://github.com/flashinfer-ai/flashinfer/pull/6360
• perf(cake_deepgemm_mega_mhc): tailpipe program for the hidden-4096 BF16 route at small split counts by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7121
• test: cover the PDL ordering contract of RMSNorm and fused add RMSNorm by @0z5a in https://github.com/flashinfer-ai/flashinfer/pull/6799
• fix(cake_rmsnorm): the backward no longer stalls behind kernels resident on other streams (SM100 / SM103 / SM107) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7131
• perf(cake_qsa_nvfp4): sm_80 launches the previous release's decode program on wide, long-stream grids by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7133
• feat(moe_ep): integrate Rubin automatic tuning with autotune v2 by @akaashrp in https://github.com/flashinfer-ai/flashinfer/pull/6535
• perf(cake_deepgemm_mega_mhc): weight-staging tailpipe program for the hidden-4096 BF16 route at many task waves by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7123
• refactor(cake_moe_grouped_gemm): entry points move to core (#7047) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7137
• ci: harden compile cache compatibility and job isolation by @righthandabacus in https://github.com/flashinfer-ai/flashinfer/pull/7063
• perf(cake_causal_conv1d): faster small-channel backward with bitwise-identical results by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7140
• perf(cake_gdp): load the token row and gate on warp 3 in the T = 1 lanerow decode programs on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7138
• perf(cake_dsa): one-launch forward for small NoPE index tables; with grad by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7139
• perf(cake_qsa_nvfp4): resolve the launch plan once per shape and launch with one stream context per call by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7141
• perf(cake_comm): fused-norm forms and row-count bands for the FP4-transport TP all-reduce by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7142
• perf(cudnn): build only saved GEMM winners on startup by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/6618
• test(cake): vendored build gate and lm_head standalone test (#7047) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7144
• feat(autotuner): add opt-in individual-call graph timing by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/6959
• fix(cudnn): support dense KV decode without page tables by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/6619
• perf(cake_deepgemm_mega_mhc): evict-first y_bf16 stores in the hidden-4096 programs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7146
• feat(qsa_sparse_paged_attention): backend="auto" default runs the Cake kernels on SM80 through SM90 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7147
• feat(autotuner): add context-scoped managed cache ownership by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7148
• perf(cake_nvfp4_svdquant): swap-AB split-K program on sm_100a and sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7151
• fix(autotuner): validate distributed tuning phases before profiling by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7152
• feat(moe): extend Frost low-latency FMA to FP16 and SM120 by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7153
• feat(comm): add a PCIe/RDMA Ulysses all-to-all backend by @triple-mu in https://github.com/flashinfer-ai/flashinfer/pull/4876
• feat(autotuner): report replay coverage and require cached winners by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7150
• fix(tests): set _tuning_state on the hand-built SM107 staging buffer by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7159
• fix(autotuner): prepare delay helper before candidate warmup by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7161
• perf(cake_msa): add L2 evict-first paged BF16 decode for Blackwell MSA by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7160
• [Autotuner] Add controlled graph selection-quality study by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7155
• perf(cake_all_gather_matmul): SM100a SM push band at world size 8 and spill-split pair at world size 4 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7145
• feat(moe): add per-tensor FP8 and Hopper FMA to Frost by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7162
• deslop(cake_dsa): with grad, fold the natural-layout clones, share the kernel preamble and binding blocks, drop dead glue by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7143
• perf(cake_kda): with grad; hints-off fused gradient from 16 heads and the 64-column token-row saving forward on SM107 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7158
• perf(cake_dsa_indexer): far_sample scan scalar, predicated slot reservation and per-program register-usage flags by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7171
• fix(moe): preserve valid loop bounds for cuTile wide indexing by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7168
• perf(cake_gdp): read the split-V T = 1 state cells straight into registers on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7156
• feat(moe): add E4M3 weight-only BF16 activation to Frost by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7166
• deslop(cake_deepseek_fused_routing): drop the 117 explicit kernel instantiations by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7172
• feat(cake_tp_moe_finalize_rmsnorm): deliver the TP4 programs on sm_107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7173
• feat(moe): add DeepSeek block FP8 to Frost low latency by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7174
• feat(cake_gdn): context-parallel packed GDN auto grad with precise cross-rank state passing, native grouped Q/K heads and per-document zigzag for Qwen hybrid GQA/GDN with grad by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7169
• feat(cake_vsa_sm90): add grouped-query (GQA) variant of the variable block-sparse attention forward by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7167
• feat(moe): extend Frost low-latency to four-bit weight-only formats by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7184
• docs(prefill): refresh the Blackwell ragged d192/128 auto-order note with cuDNN 9.27 data by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7154
• feat(moe): extend Frost weight-only decode to FP16 by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7186
• feat(moe): add Frost PowLU and SM120 W8 route reuse by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7189
• perf(moe): extend bounded SM120 BF16 TC decode through eight tokens by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7190
• perf(cake_kda): auto grad, ship the BF16x3 half-width transfer walk on the context-parallel route by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7176
• feat(cake_muse_spark_norm): deliver the sm_107a programs and cache launch plans per call signature by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7177
• perf(cake_gdp): issue the dense v / beta row loads at the k load site in the one-row-per-thread T = 1 decode kernels on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7178
• perf(cake_nvfp4_svdquant): L2-promoted 64-row program for M 129 K 3072 on sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7183
• feat(cake_vsa_sm90): batch 8 V^T chunks in the FP8 grouped-query shared-transposer queue program by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7185
• perf(cake_dsa_indexer): 16384-slot candidate buffer floor and an early next-tile probe in the wide scan programs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7187
• fix(moe): reserve W4A16 scratch for direct routes before graph capture by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7195
• perf(moe): add grouped BF16 TC decode on Hopper and SM120 by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7194
• perf(cake_mhc): faster four-stream mHC (hidden size 4096) with grad on SM100 / SM103 / SM107 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7198
• perf(cake_qsa_nvfp4): fold the block-scale rebias into the E2M1 tables and hold the table words per tile by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7175
• fix(cake_muse_spark_norm, cake_tp_moe_finalize_rmsnorm): raise the sm_107a min_cuda floor to 13.4 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7199
• perf(cake_rmsnorm): faster backward pairs on SM100 / SM103 / SM107 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7200
• feat(cake_qsa_nvfp4): tcgen05 decode programs for SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7205
• perf(cake_rubin_mla_nvfp4_decode): eight-warp split-KV merge and one-tile wave fill for Rubin (sm_107a) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7206
• feat(cake_mhc): with grad; accept run-time rms_eps / hc_eps on the shipped programs; key sinkhorn_iters by the delivered count (#7046) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7208
• perf(cake_gdp): run the many-wave T = 1 decode rows on the one-wave-of-5 program with the dense row preload and retire the two-heads-per-CTA program on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7201
• perf(cake_mxfp8_megamoe_ep16): CTA-scope weight-tile publication and deeper rings by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7210
• perf(cake_fused_norm_combine): four token-count programs for the eight-peer fused norm-combine by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7270
• perf(cake_all_gather_matmul): SM100a CTA-pair route for 512-row K-major calls at world size 4 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7192
• perf(cake_minimax_h3): K = 96 NVFP4 out-projection mainloop on SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7271
• perf(cake_mhc): cut the host time of the mHC with-grad entry points (#7046) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7272
• perf(cake_mhc): four-stream mHC (hidden size 4096) with grad keeps its speed next to a resident side kernel on SM100 / SM103 (#7046) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7275
• perf(cake_softmax): faster Blackwell cached-cluster softmax and a persistent many-row route by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7277
• perf(cake_fp4_all_reduce): cheaper scale-factor stores in Cake fused RMSNorm by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7276
• perf(cake_minimax_h3): fused MXFP8 QKV GEMM pre-attention on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7279
• perf(cake_mega_moe): faster dispatch, routing, unpermute and combine for the BF16 rank-major EP8 layer by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7278
• perf(cake_blackwell_bf16_fp4): faster SM100/SM103 BF16 x FP4 GEMM kernels for small and large M by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7282
• perf(cake_kda): with grad; fused gate epilogue on SM103, segmented 16-head rows on SM103 / SM107, two-level segment carry on SM100 / SM107, saved fold tensors on all three by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7274
• perf(cake_dense_projection_gemm): whole-tile tile resolve programs for the SM103a 128x256 input-gradient rows by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7283
• feat(cake_minimax_h3_dense_attention): generated programs for sm_90a, sm_100a, sm_103a, sm_120a and sm_121a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7291
• perf(cake_dsa_indexer): 32768-slot candidate buffers for short units on sm_103a and a re-tuned register split in the sm_103a wide scan programs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7295
• fix(cake_batch_deepgemm_fp8): use an even n_l2_group on the K=7168 expected_m<=1 serving route by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7296
• perf(cake_gdp): preload the dense row in the one-row-per-thread T = 1 decode program on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7297
• feat(cake_tinygemm2_tcgen05): add tcgen05 tinygemm_bf16 programs for sm_100a/103a/107a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7302
• feat(moe): extend grouped Frost decode to SM80 and typed activations by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7204
• perf(cake_sage): faster SM120 block-sparse Sage attention by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7303
• perf(cake_minimax_h3): 256 x 448 MXFP8 FC1 with staggered accumulator halves in the MLP block on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7304
• feat(cake_qsa_nvfp4): tcgen05 decode programs for SM107 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7284
• perf(cake_trtllm_moe_allreduce_union): latency-ordered one-shot and two-shot schedules for SM100/SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7298
• perf(cake_softmax): shared-tile Blackwell cached-cluster softmax for wide multi-wave rows by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7308
• perf(cake_nvfp4_mla_decode): faster SM100 / SM103 decode with a native E2M1 x E4M3 dequant multiply and a shorter softmax chain by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7306
• deslop(sm110): fold the ratio-clone tree units into body headers; drop dead registry leaves and NEED selectors by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7293
• deslop(cake_rmsnorm_train): fold duplicate and per-arch kernel pairs, drop unread registry keys, collapse dead guards by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7307
• perf(cake_gated_act_mxfp8): faster fused SwiGLU + MXFP8 quantization forward and backward (SM100 / SM103 / SM107) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7305
• perf(cake_router_gemm): per-shape schedules for the router GEMM on SM100/SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7287
• perf(cake_mega_moe): staging kernel, arrive-only barriers and route-in-dispatch for the BF16 rank-major EP8 layer by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7300
• perf(cake_warp_decode): retune SM100/SM103 schedule selection by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7292
• perf(cake_fused_norm_combine): pipelined owner program at lag 3, final store 4 steps behind by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7285
• perf(cake_minimax_h3): SM120 FP8 varlen attention: fused statistics, unit-scale block-scaled MMA, split K/V refill by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7289
• perf(cake_mxfp8_megamoe_ep16): one-fence NVLink barrier, per-warp dispatch, balanced static partition by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7299
• perf(cake_minimax_h3): faster NVFP4 out-projection on SM120 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7309
• perf(cake_kimi_k3_tp12_tail): route 256-960 token tails through the early-shared-scatter forms by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7310
• perf(cake_blackwell_bf16_fp4): faster tcgen05 M128 pair programs on SM100/SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7311
• feat(moe): extend grouped Frost decode to FP8 and block-FP8 on Hopper and SM120 by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7212
• perf(cake_kimi_k3_latent_moe): evict-first decode weights and a claimed-item resident TP1 tail GEMM by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7312
• perf(cake_comm): pipelined, topology-weighted PCIe CE ring all-reduce on SM120 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7301
• feat(cake_qsa_nvfp4): mma.sync decode programs and the merge for SM120 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7313
• perf(cake_minimax_h3): N-split tail for the NVFP4 out-projection GEMM on SM100/SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7314
• perf(cudnn): cache prefill shape-override version support by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7317
• docs(cake_kimi_k3_tp12_tail): the fused K23 tail serves up to eight tokens by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7316
• feat(cake_deepgemm_mega_mhc): hidden-4096 mHC inference programs for SM100 and SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7321
• feat(cake_qsa_nvfp4): persistent-walk decode for SM103 two-wave grids and the tcgen05 one-tile fused merge by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7324
• feat(cake_fused_moe): BF16 × NVFP4 W4A16 split fused-MoE kernels for sm_100a/sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7315
• fix(cake_minimax_h3): reject SM120 varlen attention inputs whose offsets overflow 32-bit indexing by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7320
• perf(cake_fused_norm_combine): packed BF16x2 peer sums, packed norm conversions, early stream loads and a one-trip pipelined token body by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7330
• fix(cake_nvfp4_mla_decode): ship build.json in the wheel and size test workspaces on the current device by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7327
• perf(cudnn): avoid Stream object construction in warm attention calls by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7331
• perf(cake_nvfp4_attention): faster SM103 kernel with a tile-boundary overlap and P in tensor memory by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7328
• perf(cake_minimax_h3): SM120 FP8 varlen attention: early PV hand-off and next-unit prefetch by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7334
• perf(cake_minimax_h3): faster FP8 out-projection on SM120 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7333
• perf(cake_grouped_mxfp8_quantize): select the bulk-copy and row-tiled programs by shape by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7322
• perf(cake_dcp_alltoall): faster DCP all-to-all CP2 / CP4 kernels on sm_100a and sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7340
• perf(cake_muse_spark_norm): PDL route for rmsnorm_sigmoid_gate with per-window launch forms by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7341
• perf(cake_nvfp4_mla_decode): faster SM100 / SM103 decode with a paced dequant transform and a shorter start chain by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7343
• perf(cake_mxfp8_megamoe_ep16): route terms returned by relaxed system-scope stores, later FC1 prefetch, evict-first weights by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7323
• feat(cake_qsa_nvfp4): the SM120 whole-tile decode, selected by the per-CTA tile stream by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7345
• perf(cake_selective_state_update): faster programs for every variant on sm_100a and sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7319
• perf(cake_mok): owner-rank load bins in the count kernels, lower host launch overhead and an early backward scratch release by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7326
• perf(cake_rubin_mla_nvfp4_decode): two epilogue staging tiles per correction warp for Rubin (sm_107a) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7329
• deslop(cake_hopper_msa): one binding per role, fold 38 clone kernels, add a device header, drop dead code by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7332
• perf(cake_warp_decode): linear-grid K256 FC2 for the H4096/E512 shard rows by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7339
• deslop(cake_mamba_ssd_combined): fold the 21 mask-only sm_107a renders and drop the dead export guard by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7349
• perf(cake_mega_moe): flag-in-data path for up to 32 rows, pushed route codes and post-wait unpermute reads for the BF16 rank-major EP8 layer by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7348
• perf(cake_mxfp8_megamoe_ep16): pushed routes and rows, direct rank-barrier wait, async-proxy fences by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7350
• perf(cake_nvfp4_sparse_mla_decode): tcgen05 programs for sm_100a and sm_103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7335
• perf(cake_grouped_fp8_gemm): swap-AB block-scaled programs for packed UE8M0 scales on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7342
• perf(cake_kimi_k3_attn_res): fence-free K2 register-carry, two-CTA-per-SM and retuned persistent programs (SM100/SM103) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7338
• perf(cake_fused_norm_combine): one-token program, packed norm in the owner programs and an early-load pipelined band by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7352
• fix(cake_causal_conv1d_train): raise the sm_107a min_cuda floor to 13.4 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7354
• perf(cake_warp_decode): five-deep-ring K256 FC2 and vector finalize for the H4096/E512 rows by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7357
• feat(cake_qsa_nvfp4): the persistent walk serves compute capability 10.7 within a three-wave bound by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7353
• perf(cake_kimi_k3_fused_router): faster Kimi-K3 fused router programs on every route for SM100/SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7355
• Add standard Blackwell MegaMoE to the next CuTe DSL backend by @akaashrp in https://github.com/flashinfer-ai/flashinfer/pull/7001
• perf(cake_mm_fp4): faster per-token NVFP4 quantizer and GEMM programs on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7356
• perf(cake_all_gather_matmul): SM103a last-pass N-split twin for 1025-row bf16 K-major calls at world size 8 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7360
• perf(cake_gated_act_mxfp8): narrow tiles for small problems and four rows per thread for large ones by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7346
• perf(cake_dense_projection_gemm): paced output-store programs for the SM103a 5120-column input-gradient rows by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7361
• fix(cake_kda): N16 aux hand-off barriers on a one-owner ring of ten slots by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7358
• perf(cake_softmax): per-tile turnover ring for the Blackwell persistent softmax by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7359
• feat(cake_nvfp4_attention): add Cake NVFP4 attention for SM100, SM103 and SM120 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7362
• docs: correct managed autotune cache examples by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7164
• perf(cake_bf16_bmm_swap_ab): tcgen05 swap-AB programs for decode-row bmm_bf16 on SM100/SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7351
• perf(cake_minimax_h3): N-split tail for the BF16 and MXFP8 out-projection GEMMs on SM100/SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7363
• perf(cake_sage): SM120 block-sparse Sage attention computes P's first half ahead of the rescale vote by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7364
• fix(cake_kimi_k3_fused_router): reject arm M launches below 128 plan-owner CTAs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7365
• [Attention] Reuse bounded cuDNN prefill plans and gate Blackwell paged auto by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/6231
• deslop(cake_fp4_allreduce): drop 5 unreachable prefetch programs, fold 14 _pf twins, share 10 bindings, trim registry by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7366
• perf(cake_grouped_fp8_gemm): store16 programs for short expert runs on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7367
• fix(cake_minimax_h3): SM120 FP8 varlen attention: proxy fence before each K / V stage release by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7369
• perf(cake_tinygemm2_tcgen05): unroll narrow tile and retune its windows by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7370
• perf(cake_dsa_indexer): a third gate and emission warpgroup for the wide scan programs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7371
• feat(cake_fp8_attention): add Cake INT8 / FP8 attention for SM90 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7373
• perf(cake_kimi_k3_attn_res): held-release and fenced K2 register-carry routes, BF16-add snapshot-write route (SM100/SM103) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7374
• perf(cake_kda): with grad; saved WY representation on the segmented 16-head rows of SM103 / SM107, two-level segment carry on SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7375
• [Attention] Admit proven full paged prefill to Blackwell cuDNN auto by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7377
• perf(cake_blackwell_bf16_fp4): faster split-K cluster programs on SM100/SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7376
• perf(cake_nvfp4_mla_decode): QK-ahead decode for long SM100 plans and a faster split-KV combine by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7385
• perf(cake_qsa_nvfp4): compute capability 10.0 / 10.3 keep the split form on every host by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7381
• perf(cake_trtllm_moe_allreduce_union): shorter first-token load chain and interleaved two-shot reduce-scatter for SM100/SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7387
• perf(cake_mm_fp4): ring-overlay 128-wide one-CTA GEMM tile on SM100, and dispatch fallback by SM count by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7380
• perf(cake_grouped_fp8_gemm): direct-store and expert-aligned FP32-scale programs and a layout-aware route rule on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7386
• perf(cake_minimax_h3): faster SM120 out-projection on RTX PRO 6000 Blackwell by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7389
• fix(cake_mxfp8_megamoe_ep16): place the alias proxy fence on the rank-barrier path by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7378
• [Attention] Accept loose KV upper bounds for cuDNN paged prefill by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/7382
• perf(cake_gated_act_mxfp8): packed f32x2 sigmoid where it pays (SM100 / SM103 / SM107) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7396
• docs: explain NVFP4 MoE precision choices and link design reference by @zianglih in https://github.com/flashinfer-ai/flashinfer/pull/7135
• perf(cake_gdp): T = 1 tail-wave band rule, T = 4 gate-logit carrier and T = 8 dense-load carrier on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7393
• feat(gdn): add SM8x prefill for the gated delta rule, fused and chunk-parallel by @lesj0610 in https://github.com/flashinfer-ai/flashinfer/pull/5109
• feat(mla): add SM90 BF16 DOTS3 sliding-window decode by @DylanChen-NV in https://github.com/flashinfer-ai/flashinfer/pull/6823
• perf(cake_minimax_h3): persistent NVFP4 norm and norm-overlapped NVFP4 FC1 in the MLP block on SM100 / SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7383
• perf(cake_minimax_h3): faster FP8 / NVFP4 pre-attention on SM120 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7347
• perf(cake_kda): faster BF16-input KDA prefill with FP32 state by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7388
• perf(cake_fused_norm_combine): record-first loads in the four-CTA pipelined program and a packed readiness test by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7390
• perf(cake_grouped_mxfp8_quantize): route long rows of many-row problems to the 224-thread tiled programs by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7391
• perf(cake_kimi_k3_latent_moe): packed one-partial norm for TP8 prefill tails, 256-byte L2 promotion of the decode weight maps by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/7395

Auto-generated notes truncated to fit the GitHub release body limit. Full changelog: v0.7.2rc7...v0.7.3