github NVIDIA/cudnn-frontend v1.13.0
v1.13.0-release

15 months ago

cudnn frontend v1.13 release notes

cudnn frontend v1.13 is the preferred cudnn frontend version for cudnn version 9.11.0 and above.

New API

Introduces device descriptor, which allows for device-less compilation of cudnn graph on a target GPU. See newly added sample and documentation.

Improvements

SDPA

  • Introduced generate_stats as a replacement for is_inference, to improve clarity. When generate_stats is true, the output will contain the stats tensor. When migrating from is_inference (which is now deprecated), note that generate_stats has the opposite meaning, so pass it the negation of the bool that was passed to is_inference.

  • Improved support checks for left and right diagonal bands in conjunction with the diagonal alignment.

  • Improved error handling for large head dimension (d > 128) in sdpa bprop.

Normalizations

  • Added support for fused Layernorm with Relu and samples for Layernorm with relu bitmask dump

Others

  • Published improved SDPA training benchmarks for fp8 and fp16/bf16 graph patterns.

  • Enable int4 Weight only Quantization for matmul. See example

  • Allow block scale dequantize (required for low precision matmul) to take 2-D scale factor.

  • Allow reductions to accept deterministic as a attribute.

  • Added pybinds for block scale dequantize.

Bug Fixes

  • Fixed the sliding window attn_score_modifier function allowing it to set true negative infinity.

Don't miss a new cudnn-frontend release

NewReleases is sending notifications on new releases.