cuDNN Frontend v1.33.0 Release Notes
cuDNN Frontend v1.33.0 adds simplified sparse attention for PyTorch and JAX, Cake training operations, and prebuilt attention for SM120. Native SDPA expands on Hopper and SM120. Applications that use the experimental triangle operations should review the API changes before upgrading.
Sparse attention and training ๐
- Simplified sparse attention supports PyTorch and JAX. Training includes a dense indexer KL loss. (#2370)
- Cake adds deterministic GDN and KDA training engines, GQA and sparse attention, and DSA indexer training. Other training operations cover GEMM, SwiGLU/MXFP8, causal convolution, RMSNorm, LM-head, MoK, and mHC. (#2432, #2435, #2437, #2439, #2445)
- JAX users gain DSv4 indexer, top-k, sparse attention, and compressor operations, plus NSA and HSTU operations. The DSA JAX indexer accepts packed THD inputs. (#2165, #2336, #1777, #1769)
- Hopper gains a fused compressed DSA indexer and top-k operation and a prepared plan for repeated calls. Block-sparse attention supports per-token Q/K FP8 scales. (#2367, #2406, #2004)
SDPA forward and backward ๐
- Hopper gains native D64, D128, and D256 forward paths and paged KV forward. Dense D64/D128 backward can use a deterministic kernel. Backward also supports bottom-right causal masking. (#2185, #2426, #2431, #2535, #2753)
- SM120 gains paged decode with split-KV, paged D512 attention, and half-precision D576/V512 forward. Native backward adds half-precision and FP8 paths through D512, plus MXFP8 with optional amax. (#2396, #2415, #2418, #2422, #2423, #2424, #2425)
- SM100 and SM103 gain improvements to MXFP8 backward on SM100 and NVFP4 quantization-aware training backward on SM103. (#2688, #2102)
- Prebuilt SM120 attention exposes prefill, split-KV, and quantized-KV decode through a C++ header. The prebuilt kernels run without runtime compilation. (#1782)
Linear attention, normalization, and execution โจ
- Fused GDN decode and multi-token verification support SM100-class and SM120-class GPUs. Per-rank head counts and hidden widths support sharded models. (#2242, #2268, #2292)
- Repeated KDA and GDN calls with an eligible argument signature can replay recorded launches from C++, reducing Python dispatch work. (#2279, #2281)
- JAX gains LayerNorm and RMSNorm with native gradients. LayerNorm and RMSNorm also accept strided inputs. On SM100-class GPUs, decode can fuse residual addition, RMSNorm, and quantization. (#1812, #2263, #2570)
- C++ and Python graph builders gain user-defined reductions through
Reduce_UD. (#2218)
Compatibility and API changes
- The experimental
cudnn.triangle_attentionandcudnn.triangle_multiplicative_updateAPIs introduced in v1.32.0 have been removed. (#2267) - mHC operations now live under
cudnn.ops.mhc. Fused DSA indexer top-k lives undercudnn.ops.dsa. Simplified sparse attention, SwiGLU quantization, and BSA backward metadata operations also changed names. Callers using the earlier development names must update their imports. (#2702)
Samples, benchmarks, and tools ๐
- The model samples add serving-style requests and reuse session state across decode and verification steps. (#2711, #2741)
- Attention inference benchmarks support CUDA Graph timing and resuming interrupted sweeps. Reported inference TFLOPS count unmasked attention pairs. (#2729, #2730, #2747)
- The SDPA reproducer preserves paged KV geometry, cumulative sequence lengths, and saved forward statistics in backward reproductions. (#2731, #2748, #2752)
Source
Release source ยท Changes since v1.32.0 source
Acknowledgements ๐
Thanks to everyone who contributed commits to this release:
@0z5a, @adnios, @alfredclwong, @andreykarailiev, @Anerudhan, @Aneureka, @AshOfCat, @bastianhagedorn, @chmayer-gpu, @dinodeep, @egilliam-nv, @GarlGuo, @harshithkantamneni, @hwanseoc, @icavan, @jberchtold-nvidia, @jeremylin0303, @jhjpark, @jiayus-nvidia, @Levius-Fubuki, @liujane-dev, @LongChengNvidia, @meinie0826, @mgoldfarb-nvidia, @miyanyan, @msalasooNV, @MuuSeoTia, @p3achyjr, @pggPL, @pmdavies-nv, @pushkar-sarvam, @RomanAnders90, @romitjain, @romitjain-sarvam, @scottyokim, @shangxiaokang, @tejesh-sarvam, @thynics, @tiffany940107, @tongke6, @TorinLi, @vedaanta, @vikramchandrashekar, @vincejhan, @vishwakulkarni, @xiufanl, @xrennvidia, @YangXu1990uiuc, @yanqinz2, @yanzhuo607, @yekerr, @yeliu-oss, @yihuawei, @zach-ye0, @zheyang0825, @zhibinz-nv, @ZhiyuLi-Nvidia, @zhongbozhu.