onnxruntime 1.30.0

ONNX Runtime v1.30.0

on Python PyPI

ONNX Runtime 1.30.0 expands generative AI inference, improves CPU and GPU performance, adds Go bindings, and strengthens runtime reliability. These notes cover changes since ONNX Runtime 1.29.1.

Highlights

• Expanded CUDA inference support with variable-length causal convolution for continuous batching, speculative decoding in paged XQA, and INT4 paged KV caches with per-channel scales (#32168, #32340, #32515).
• Improved WebGPU PagedAttention, added GPT-OSS support and INT8 KV-cache block quantization, and extended convolution optimizations (#31727, #32277, #32284, #32420).
• Added fused CPU LinearAttention kernels for AVX-512, Arm64 NEON, and SVE, plus AVX2 LayerNorm/RMSNorm acceleration (#31674, #31973, #32178, #32356).
• Added Go bindings for the ONNX Runtime C API and DeepSeek Engram contrib operators (#29615, #32268).

Announcements & Compatibility

• FP4 QMoE kernels are now enabled by default in CUDA builds, with Windows build support added in this release. Source builds can opt out with -Donnxruntime_USE_FP4_QMOE=OFF (#32096, #32163).
• CUDA fpA-intB builds now default to a compact kernel set for FP16 activations, INT4/INT8 weights, scale-only quantization, and block_size=32. Set -Donnxruntime_USE_FPA_INTB_GEMM_FULL=ON when building from source to retain the full kernel set, including BF16, zero-point, bias, larger-block-size, and native Hopper variants (#32324).
• CPU FP16 Gemm and MatMul execution is gated on hardware acceleration. CPU-assigned FP16 nodes without a matching kernel now fall back to FP32 (#32301, #32197).
• WebGPU plugin EP packaging now supports Linux AArch64. Plugin versions were advanced to WebGPU 0.4.0 and CUDA 0.2 (#32287, #31960, #31970).

Security & Reliability

Model Loading, Memory, and Input Validation

• Limited nested model-graph depth and canonicalized external-data locations to harden model loading (#32344, #32135).
• Added checked rounding for BFC arena allocations and fixed prepacked-weight reference lifetimes (#32010, #32040).
• Strengthened shape, rank, and parameter validation for Split, Scan, GatherND, ScatterND, SpaceToDepth/DepthToSpace, Crop, Conv, Normalizer, and pooling (#29461, #31668, #32034, #32039, #32076, #32157, #32160, #32161, #32345, #32349).
• Hardened generation and attention input handling, including attention-attribute narrowing, BifurcationDetector inputs, generation subgraph shapes, and QEmbed segment inputs. BeamSearch buffer expansion now uses dynamic shape storage (#31648, #31701, #32009, #32078, #32144).
• Validated TreeEnsemble node references and bounded subtree comparison, rejected non-finite CPU RoiAlign coordinates, and required ImageScaler bias to match the channel count (#32031, #32043, #32011, #32002).
• Added an allowlist of safe LoRA adapter parameter data types, validated MatMulFpQ4 shape inputs, and checked MLAS blockwise quantization/dequantization index ranges (#31682, #32032, #32007).

GPU Bounds and Resource Lifetimes

• Hardened CUDA indexing and buffer-size arithmetic in MatMulNBits, RemovePadding, RotaryEmbedding, SparseAttention, Whisper beam search, NMS, QDQ, and GatherElements (#31643, #31994, #31995, #31996, #31998, #32014, #32029, #32030).
• Fixed overflow in CUDA reduction scans and Softmax offset arithmetic, and handled zero-sized outputs in CUDA random-generator kernels (#32137, #32330, #31997).
• Fixed CUDA MultiHeadAttention shared-cache scratch lifetimes and kept CudaAsyncBuffer staging storage alive across CUDA graph replay (#31968, #32121).
• Fixed WebGPU out-of-bounds subgroup-matrix loads for partial tiles, zero-initialized writable device-allocator buffers, and rejected foreign GPU handles in built-in data transfers (#32364, #32063, #32317).

Dependencies and Tooling

• Upgraded Protobuf to 33.6 and refreshed Python documentation dependencies, including an ONNX security-related update (#29906, #32190, #32424).
• Updated JavaScript dependencies including js-yaml, joi, fast-uri, and the Next.js end-to-end fixture (#32397, #32486, #32488, #32505, #32508).
• Pinned GitHub Actions to full-length commit SHAs and strengthened packaging infrastructure with authenticated package feeds and NPM network isolation (#32176, #32005, #32440).

New Features

Core APIs & Runtime

• Added Go bindings for the ONNX Runtime C API (#29615).
• Extended memory importing with host-pointer support and added access to preallocated outputs through KernelContext::GetPreallocatedOutput (#29726, #32089).
• Added packed-attention workspace recipes and estimates, and made workspace input-shape handling aware of optional inputs (#32283, #32321, #32312).
• Added DeepSeek Engram contrib operators, EngramGate and NGramHashMapping, and expanded kernel coverage for Qwen-3.5 operators (#32268, #32106).

Plugin Execution Providers

• Added LoRA adapter support with plugin EP allocators, preserved custom allocators during EP registration, and reset plugin stream chunks before release (#32221, #32272, #31983).
• Fixed CUDA plugin device discovery on WSL and improved Windows ARM64 packaging (#32517, #32355).

Execution Provider Updates

CUDA EP

Attention and Decoding

• Added INT4 paged KV caches with per-channel scales and an is_causal attribute to PagedAttention (#32515, #32225).
• Extended paged XQA with speculative decoding, query-to-KV head group size 6, head size 256 including FP16 caches, and native block tables for 128-token pages (#32340, #32108, #32229, #32263, #32127).
• Enabled split-KV for paged FlashAttention decode and improved PagedAttention dispatch diagnostics (#32102, #32099).
• Added VarlenCausalConvWithState for continuous batching and compact variable-length causal-convolution state updates (#32168, #32290).
• Added a compact GatedDeltaNet operator and BFloat16 support for CUDA GatedDeltaNet (#32282, #32307).

MoE and Quantized Matrix Multiplication

• Added an opt-in FP8 DeepGEMM MoE decode path for supported fixed-shape QMoE workloads on Hopper GPUs (ORT_QMOE_FP4_DEEPGEMM=1, default off). This path is disabled on Windows (#32122, #32485).
• Bounded QMoE workspace with configurable row tiling and FP8 weight-dequantization scratch memory by tiling over output columns (#32097, #32129).
• Vectorized NVFP4 weight dequantization for prefill, tuned NVFP4 GEMV tiling for Qwen multi-token prediction, and extended speculative-decode GEMVs to 64 rows (#32128, #32140, #32289).
• Tuned FP4 and FP8 GEMV scheduling for 48-SM SM121 GPUs, including FP8 KSplit32 scheduling, and improved FP8 GEMV residency for grids just beyond two blocks per SM (#32408, #32409, #32433).
• Added an opt-in split-K GEMV path for small-N FP16 MatMul shapes and refined fpA-intB GEMV support checks (#31478, #32338).

Operators

• Improved TopK, ArgMax, and ArgMin performance for wide last axes, and accelerated low-lane INT64 CumSum (#32404, #32092, #32238).
• Added a single-memcpy Slice fast path for contiguous subregions and removed pinned-buffer use from Split and Concat fast paths (#28902, #32410).
• Registered BF16 ReduceMean kernels and fixed ScatterElements reduction dispatch by element type and signed-zero handling in Abs (#32326, #29879, #31477).

WebGPU EP

• Improved PagedAttention and added PagedAttention metadata, GPT-OSS support, and INT8 KV-cache block quantization (#31727, #32277, #32284).
• Added INT64 Gather support and optimized MatMulNBits wide tiles with subgroup shuffle (#31714, #31703).
• Extended convolution fusion with eight additional activations, fused activation handling in the im2col path, and uniform-based activation parameters for Conv/MatMul (#32117, #32185, #32116).
• Reused subgroup-matrix MatMul for pointwise convolution and added convolution-weight prepacking for the im2col-matmul path (#32304, #32420).
• Enabled GELU and BiasGELU fusion, added transpose-optimizer handling for Elu and contrib GELU variants, and enabled LayerNorm fusion to fix FP16 inference correctness (#32053, #32118, #32294).
• Added subgroup-size control, selected subgroup size 32 for subgroup-matrix MatMul/Gemm, and enabled the subgroup-matrix path in WASM builds (#32056, #32306, #32269).
• Scaled Dawn pipeline-compilation workers with CPU count, vectorized Split when all output segments are vec4-aligned, and selected pooling paths based on occupancy (#29820, #32251, #32313).
• Exposed safe graph-capture I/O in Python, added a robustness provider option, and enabled forwarding of onnxruntime_perf_test -i options to WebGPU (#32074, #31971, #32316).
• Fixed synchronization for user-provided GPUDevice instances, corrected MatMul pipeline-cache keys and the 1D-dispatch shader fast path, and changed copy_tensors misuse to report errors instead of terminating the process (#32259, #32048, #32343, #32315).

WebNN EP

• Added SkipLayerNormalization support and corrected output-rank validation and fallback data-type support checks (#32377, #31708, #32067, #32293).

TensorRT & DirectML

• Fixed TensorRT shape-value handling (#32415).
• Validated kernel_shape and output_padding lengths during DirectML kernel setup (#31999).

CPU & Core Optimizations

MLAS and CPU Kernels

• Added fused LinearAttention kernels for AVX-512, Arm64 NEON, and SVE (#31674, #32178, #32356).
• Added AVX2 LayerNorm/RMSNorm kernels and registered BFloat16 LayerNorm/RMSNorm kernels on the CPU EP (#31973, #31974).
• Added Arm SVE i8mm INT8 QGEMM kernels and enabled the SBGemm fast-math path on Darwin Arm64 (#31146, #32152).
• Improved NCHWc convolution thread utilization, added HardSwish fusion for MobileNetV3 models, and introduced an AVX-512 16-wide Erf kernel and NCHWc reorder improvements for MobileCLIP-S0 (#31660, #31957, #31958).
• Optimized INT4 weight prepacking on CPU, rejected KleidiAI Q4 prepacking with dynamic scales, and fixed Arm64 SymmQgemm INT16 overflow (#31690, #32068, #32057).
• Fixed FP16 QuantizeLinear rounding, prevented CPU TensorScatter index overflow, serialized ScatterND string updates, and widened Compress loop counters (#32452, #32012, #32033, #32008).
• Improved edge-case handling for empty LpNormalization inputs, zero-element BiasGelu/FastGelu, extreme Trilu diagonals, and empty reduction axes (#32020, #31698, #32013, #32156).

Graph, Optimizer, and Runtime

• Hardened graph optimizers against model-supplied indices and invalid fusion inputs, including MatMulNBits dequantization block sizes, GQA projection shapes, and Slice starts rank (#31670, #31678, #32018, #32044).
• Prevented overlapping MatMulIntegerToFloat fusions, skipped overridable-initializer fusion, fixed FuseInitializersTransformer consumer-input lookup, and avoided fusing identity transposes into Gemm (#32038, #32143, #32426, #32435).
• Fixed optional zero-point input handling in QDQ, empty-initializer axis scaling, and NodeAttrHelper string-default lifetimes (#32051, #32138, #32019).
• Released external-data loaders after graph initialization, added portable random-access file reads to Env, and clarified how external-initializer paths interact with EP context paths (#32502, #32503, #32442).
• Avoided redundant ONNX schema registration when static registration is enabled, updated cpuinfo with thread-safe deinitialization, and prevented telemetry crashes in shell-less containers (#32353, #32300, #32226).
• Reduced the active-session mutex scope around ETW callback registration and filtered out display adapters using the Microsoft Basic Render Driver (#32000, #32006).

Language Bindings & Web

• Retained Python asynchronous-run resources and pinned C# RunAsync arguments until completion (#32041, #32015).
• Fixed Java provider-option value cleanup, validated Rust tensor element types, and made Rust string-tensor output extraction safe (#31828, #32035, #32045).
• Avoided duplicate N-API cleanup-hook registration in Node.js and added FP16 support to the JavaScript WebGPU Conv3DNaive shader (#32469, #32357).
• Fixed WinML image-dimension overflow (#32046).

Build, Packaging & CI

• Upgraded CUTLASS to 4.7 and cuDNN Frontend to 1.27, and enabled compact fpA-intB kernel builds by default (#32111, #32324).
• Fixed Windows CUDA 12.9 SM120 compilation, CUDA 13 CCCL include paths in plugin builds, Windows DeepGEMM build errors, and PagedAttention builds without FlashAttention (#32114, #32392, #32485, #32327).
• Added D3D12 Agility SDK support for Dawn WebGPU builds while excluding WebGPU plugin EP builds from Agility SDK use (#32253, #32380).
• Added BTI support to MLAS AArch64 assembly, restored runtime vector-extension checks on RISC-V, and disabled POWER MLAS kernels when VSX is unavailable (#32070, #32406, #32389).
• Improved CUDA plugin ARM64 build parallelism and packaging tests, and based plugin development versions on commit timestamps (#32165, #32399, #32072, #32095).
• Separated provider compilation from archive creation, corrected Python-binding delay-load linkage, and fixed Linux minimal-build dependencies and spurious Windows Spectre-mitigation warnings (#32162, #31637, #32491, #32516).
• Expanded PagedAttention CUDA test coverage and head-size-256 paged XQA coverage, corrected GatedDeltaNet test verification, and removed a Hugging Face download dependency from a dynamic-quantization test (#31687, #32262, #32329, #32443).
• Added WGSL template tests to CI and enabled WebGPU CI on WebGPU plugin EP release branches (#32214, #32090).

Contributors

Thanks to our 57 human contributors for this release!

@4n4ny4, @apsonawane, @arnej27959, @baijumeswani, @bmehta001, @chilo-ms, @crvineeth97, @daijh, @danfiedler-msft, @danielsongmicrosoft, @dannyota, @DKAIN-py, @edgchen1, @ericcraw, @eserscor, @fanchenkong1, @hanbitmyths, @hariharans29, @hdharpure9922, @Honry, @jambayk, @javier-intel, @jchen10, @jiafatom, @jnagi-intel, @justinchuby, @kadyrbekovhamit-cyber, @kunal-vaishnavi, @Lapis0x0, @LOGO127, @Manogna-Sree, @martin-klacer-arm, @mei1127, @miaobin, @mirounga, @miyanyan, @MohamedElashri, @mustjab, @Nikhi00718, @Noperi0r, @Novestars, @pkubaj, @preetha-intel, @qjia7, @rvandermeulen, @sanaa-hamel-microsoft, @skottmckay, @sushraja-msft, @swetha097, @sylvesterkaczmarek, @tianleiwu, @titaiwangms, @toothache, @xadupre, @xhcao, @xiaofeihan1, @Zestion

Full Changelog: rel-1.29.1...rel-1.30.0

Release highlights were prepared with AI assistance.