This release expands model and operator support, improves performance and numerical stability, and adds provider configuration and shader-cache controls.
Highlights
Model and operator support
- Added
Or,Xor,Round,Sign,IsNaN,IsInf,Softsign,LpPool, andGlobalLpPoolkernels. (#33680, #33679) - Added
QuantizeLinearwith signed and unsigned 8-bit and 4-bit outputs, and 4-bitReshape. Supported opsets andprecisionrestrictions apply. (#33683, #33792) - Added FP16 support for
GRUandLSTM, int32 support forRelu, and int8 support forGather. (#33678, #33789, #33790) - Added signed int8 and int64 support for
Abs,Neg,Relu, andSign. Int64 kernels require theenableInt64provider option. (#33958) - Added
MatMulBlockQuantizedFp8Weightsupport for FP16 activations, E4M3FN FP8 weights, and FP32 block scales. (#33074) - Added automatic
MatMulNBitsQLoRAfusion for supported models, using 4-bit block-32 base weights and 8-bit block-32 rank-32 or empty adapters.MatMulNBitsalso supports empty packed weights. (#33370, #33559) - Extended
GatedDeltaNetto support per-key-dimension decay. (#33817) - Added
ConvStateReplayandGatedDeltaNetStateReplayfor reconstructing speculative-decoding state. These operators do not support graph capture;GatedDeltaNetStateReplayuses FP32 state and capsule backing. (#33493, #33213) - Added
PackedSparseAttentionIndexerMergeto combine sparse-attention index sets. (#33224)
Performance and configuration
- Added an Intel-specific dense flash-attention prefill path for unquantized KV caches. (#32702)
- Added subgroup-matrix acceleration for eligible Intel FP16, group-1, channels-last convolutions. Other configurations use existing paths. (#32252)
- Improved
MatMulandMatMulNBitspaths, including NVIDIA and AMD subgroup-matrix kernels and Qualcomm DP4A tuning. (#33639, #32796, #33560) - Reduced transfer overhead through batched input uploads, direct queue writes for eligible small CPU-to-GPU uploads, and batched compatible streamless GPU-to-GPU copies. (#33491, #33551, #33567)
- Extended the opt-in
enableMatmulFp32Accumulationoption to FP16MatMulandGemm. It remains off by default; enabling it can bypass FP16-only subgroup-matrix paths. (#33057, #32630) - Added per-session
enableFeaturesanddisableFeaturesoptions, initially supportingsubgroupMatrix. Disabling it selects existing fallback kernels. (#33481) - Added native
dawnCacheInterfacecallbacks for caching Dawn shader and pipeline blobs, not model weights. Configure the interface before ORT creates the device; it cannot be attached to a host-supplied device. Later sessions may inherit the shared context's cache configuration. (#33936, #33999)
Correctness and packaging
- Improved numerical stability for
LayerNormalization, FP16 reductions,InstanceNormalization, and several elementwise functions. (#33365, #33682) - Fixed graph capture and replay of GPU buffer copies and explicit buffer clears. (#33362, #33509)
- Synchronized Session command recording across execution, graph replay, and CPU I/O operations. Applications must still avoid overlapping I/O binding and
Runon the same Session. (#33763) - Fixed shader-cache collisions across operator variants and a deferred pipeline callback lifetime issue after failed waits. (#33242, #33209)
- Fixed scalar-bias
Gemm, vectorizedPowbroadcasting, three-channelConv2dMMindexing, and empty pooling outputs. (#33664, #33348, #33432, #33668) - Bundled the preview D3D12 Agility SDK in Windows Python wheels, the NuGet package, and Foundry Local zip packages. (#32795)
Runtime Compatibility
The minimum ORT version is 1.24.4. Concurrent independent Sessions are enabled on ORT 1.28.3 and later in the 1.28 series, 1.30.1 and later in the 1.30 series, and 1.31 or later. Other supported versions, including 1.29.x, use serialized compatibility. In serialized mode, callers must serialize all WebGPU operations on the same device, including Session creation/destruction, allocation, I/O binding, transfers, and Run; the plugin does not enforce this serialization. Concurrent mode does not permit overlapping I/O binding and Run on the same Session (#33082).
Known compatibility caveat: ORT 1.26.0 was reported to reject the plugin during kernel creation because of its host-side plugin-version check. Forcing serialized execution does not resolve that restriction (#33082).
Activating the bundled Windows preview Agility SDK requires a compatible Windows version and loader update, Developer Mode, and a compatible GPU driver. See the Agility SDK guidance.
Contributors
Thank you to @aciddelgado, @adrastogi, @apsonawane, @baijumeswani, @daijh, @edgchen1, @GopalakrishnanN, @gramalingam, @gyagp, @jchen10, @Jiawei-Shao, @justinchuby, @keshavv27, @kunal-vaishnavi, @kylo5aby, @mei1127, @mohdmahmodi, @mustjab, @prathikr, @qjia7, @sayanshaw24, @SharmaRithik, @sushraja-msft, @tianleiwu, @xenova, @xhcao, @xiaofeihan1, and @ykhrustalev.
These notes focus on ONNX Runtime WebGPU Plugin EP changes since 0.5.0; contributor acknowledgments also cover commits touching shared WebGPU EP code.
These notes were prepared with AI assistance.