cudnn_frontend v1.0 release introduces new API aimed to simplify graph construction.
[New API] In FE v1.0 API, users can describe multiple operations that form subgraph through cudnn_frontend::graph::Graph object.
Unlike the FE v0.x API, users dont need to worry about specifying shapes and sizes of the intermediate virtual tensors. See README.FE.1.0.md for
more details. For more information on historical 1.0 changes, pre-release release notes are here.
Graph class consist of three types of API, viz.
- APIs that return reference to the graph itself.
This is necessary for chaining.
These can be used for setting the global properties of the graph. Example,
graph.set_compute_data_type(...).set_io_data_type(...);
- APIs that return a shared pointer to the tensor. These are required to denote entry tensors or output of nodes which can be exit points of graph or inputs to other nodes. Example,
X = graph.tensor(...);
W = graph.tensor(...);
Y = graph.conv_frop(X,W, Conv_fprop_attributes(...));
- APIs that return a error type which is a combination of error code and error message. These APIs generally mutate the graph object, or are responsible for calling the cudnn backend API. Example,
auto error = graph.validate();
auto error = graph.build_operation_graph(handle);
[New Feature] Python bindings for the FE 1.0 API. See, Python API section in README.md for building the python bindings. Details of python
API and its kw arguments are in the README.FE.1.0.md. Python API samples are in samples/python/*.py
[New Feature] Added a compound SDPA op (both forward and back prop). More details in docs/operations/Attention.md
[New Feature] Better error reporting, where in addition to error codes, we also provide error messages which provide more information on specific cause of failure.
[Deprecation] v0.x API are now labelled deprecated and may be removed in v2.0. Consider moving to v1.0 API. If there are issues or missing features, please create a github issue.
Changes over pre-release-5:
[New Feature] Scaled_Dot_Product_Attention op now supports GQA in Fprop and bprop.
[Breaking change] Output dim and strides of SDPA fprop and bprop outputs are now mandatory. Since, the inference of output shapes are non-deterministic.
[Samples] Added samples to showcase,
- INT8 convolution (
"Conv with Int8 datatypes") - Mixed precision multiplication (
"Mixed Precision Matmul") - Simple Convolutions, MatMuls and Matmuls with simple epilogues(
matmuls.cpp, wgrads.cpp, dgrads.cpp)
[Update] The default value of cudnnNanPropagation_t has been set to CUDNN_PROPAGATE_NAN instead of CUDNN_NOT_PROPAGATE_NAN.
[Update] Have added a typedef for scaled_dot_product_flash_attention as SDPA as a convenience.
Miscellaneous updates to v0.x API and the legacy samples:
[Bug fix] Some tests were failing on Ampere GPUs because no plans with 0 size were available. This has been fixed.
[Bug fix] Median of three sampling was incorrectly sorting the results, when cudnnFind was used. This has been fixed.
[Bug fix] Thanks to @Riottomsk for pointing out the bug in port count of Pointwise mode POW in his [PR] (#49). This fix has been incorporated.
[Bug fix] Have fixed a bug in resample backprop operation, where CUDNN_ATTR_OPERATION_RESAMPLE_BWD_XDESC and CUDNN_ATTR_OPERATION_RESAMPLE_BWD_YDESC were not set correctly.
[Feature] Layer Norm API has been added. And can be used with the v0.x API.