microsoft/onnxruntime v1.28.0

ONNX Runtime v1.28.0

on GitHub

Announcements & Breaking Changes

• Upgraded to ONNX 1.22.0 and protobuf 6.33.5 (#28754, #29606, #28967). Graph optimizer opset version checks were updated accordingly (#28966).
• cuDNN and cuFFT are now optional at runtime for the CUDA EP, and nvrtc is no longer linked, which significantly reduces the required CUDA redistributable footprint (#29252, #29808, #29705, #29620).
• An experimental C/C++ API surface was introduced. OrtModelPackageApi now lives in the experimental C API and may change in future releases (#28746, #29142, #28990).
• Deprecated / removed:
  • SkipLayerNorm strict mode is deprecated (#29388).
  • The TensorRT fused causal attention kernels were removed from the CUDA EP (#29143).
  • The dynamic WGSL generator (duktape/Node) path was removed in favor of the Python wgsl-gen implementation (#29141, #28355).
  • CUDA_QUANT_PREPROCESS is off by default (#29687).
• NPM packages are now published from the CUDA 13 pipeline (#28773).
• The CUDA 12.8 package architecture list was refreshed for this release (#29711).

Security Fixes

Memory safety & input validation

• Hardened the ORT FlatBuffer model loader against malformed buffers, and removed now-redundant table offset validation (#28186, #29068)
• Fixed type confusion in raw-pointer bind_input causing an out-of-bounds write (#28839)
• Fixed out-of-bounds pointer in TensorAt for sub-byte packed types (#28973)
• Fixed arbitrary memory read, out-of-bounds dereference, and other OOB accesses in kernels (#28991, #29011, #29012, #29014)
• Validated Col2Im inputs to prevent heap over-read (#28706)
• Hardened CropAndResize against malformed crop_size tensors (#28766)
• Validated BeamSearch vocab_size against logits width (#28774)
• Fixed bounds in WhisperDecoderSubgraph::CreateInitialFeeds (#29239)
• Validated SparseAttention CSR indices/key lengths and rejected zero-dimension block_row_indices (#29015, #29242)
• Clamped derived sequence lengths and KV-cache index in CUDA GroupQueryAttention, and fixed a CPU GQA out-of-bounds read in the past-KV buffer (#29240, #29447)
• Clamped 1D attention mask_index to valid bounds (#29449)
• Validated MaxpoolWithMask kernel rank against input spatial rank (#29253)
• Rejected CUDA BERT EmbedLayerNorm/SkipLayerNorm shapes exceeding 32-bit output indexing (#29264)
• Fixed the optional-output guard in DecoderAttention/MultiHeadAttention shape inference and negative-axis handling in ExpandDims shape inference (#29268, #29448)
• Fixed TreeEnsemble target id validation and added input validation to LinearClassifier (#29293, #29060)
• Fixed DynamicQuantizeLSTM zero-point/scale validation typos (#29462)
• Handled non-trivially-copyable types in Loop/Scan output concatenation (#29397)
• Normalized bool tensor raw_data to {0, 1} on unpack (#29238)
• Addressed hardening gaps in Resize, PadFusion, and LoRA handling (#28779, #28780, #28801)
• Fixed unbounded lifetime on WithOutputTensor in the Rust bindings (#29251)

Integer overflow & allocation size

• Guarded MlasConvPrepare working-buffer products and ConvTranspose pad computation with SafeInt (#29444, #29446)
• Fixed signed-int overflow in SamplingState::Init that could cause a heap buffer overflow (#29443)
• Hardened QMoE against integer overflow and partial K tiles (#29067)
• Validated B/scales/zero-points shape in MatMulNBits::PrePack (#29445)
• Pre-checked ConstantOfShape output size against the input initializer before constant folding (#28751)
• Fixed integer overflow in RKNPU implicit bias allocation (#29249)
• Fixed WebGPU out-of-bounds reads in Pad (int64/int32 truncation), Slice, and GatherBlockQuantized (#28721, #28704, #28718)

Supply chain & tooling

• Updated protobuf to mitigate CVE-2026-0994 and bumped ONNX/protobuf to fix additional CVEs (#28967, #29606)
• Avoided shell injection in the training helper and switched Triton compile helpers to subprocess (#28776, #28775)
• Validated archive extraction paths in the transformers tooling (#28777)
• Validated and inlined external data in node tensor attributes during session initialization (#29250)
• Enabled Spectre-mitigated MSVC libraries for BinSkim builds (#29624)
• Bumped npm dependencies: shell-quote, esbuild, tmp, ws, protobufjs, js-yaml, tar, markdown-it, @babel/core (#29022, #29044, #29055, #29057, #29061, #29062, #29063, #29079, #29090, #29156)

New Features

Execution Provider ABI & Plugin EPs

• Model Package support Phase 2, plus authoring tools, schema versioning, and folding external_data into session options (#28271, #28989, #29501)
• Added an API to select the best compiled-model compatibility info from candidate strings (#28387)
• Added crypto support: applications can supply I/O callbacks to an EP, with callback and fallback helpers (#28624)
• Implemented name-based partitioning with accompanying documentation (#28903)
• Added Linux NPU discovery through sysfs accel devices (#28703)
• Relaxed CompileModel validation to accept zero-input OrtModel graphs (#28771)
• CUDA plugin EP: user compute stream with CUDA graph, kernel sync stream exposed for scratch allocation, and Windows ARM64 packages (#29221, #29244, #28896, #28789)
• WebGPU plugin EP version bumped to 0.3.0 (#29056)

Core APIs & Runtime

• Added OrtErrorCode documentation, single-sourced the values so StatusCode stays in sync, and added OrtErrorCode::ORT_DEVICE_RESET (#29018, #29065, #29748)
• Added memory statistics to profiling output (#29058)
• Added EP version logging on inference failure, in the EpDeviceUsage event, and ORT version logging (#28794)
• User-supplied external initializers are now used in place when already on the planned device (#29013)
• model_external_initializers_file_folder_path is now honored for file-path model loads (#29459)
• Added a Python API for HOST_ACCESSIBLE OrtValue allocation (#28038)

Quantization Tooling

• Added CudaQuantizer to onnxruntime.quantization (#29509)
• Registered Flatten as a Direct8Bit op in the Python QDQ static quantizer (#28340)
• Skipped MaxPool during FP8 static quantization and fixed the FP8 (FLOAT8E4M3FN) scale reference distribution (#28488, #29350)
• Added Float16/BFloat16/Float8 support in the TensorArray custom op (#28335)
• Clarified CPU parameter recommendations in the quantization docs (#28415)

Execution Provider Updates

NVIDIA CUDA EP

Attention & LLM decode

• Enabled XQA by default for FP16/BF16 GroupQueryAttention, and extended XQA decode with attention sink, sliding window, and QK-Norm support (#29046, #29162, #29177, #29186)
• Upgraded cudnn_frontend to 1.24 and enabled cuDNN SDPA for MHA/GQA (#28849)
• Added decode-optimized LinearAttention (GatedDeltaNet) kernels (#28985)
• Optimized FlashDecode split planning for local-window GQA and fixed Flash/Lean attention split heuristics (#29161, #29554)
• Updated the GroupQueryAttention contrib op documentation (#29173)

MoE & quantized GEMM

• Prepacked int4/int8 QMoE expert weights in the PrePack hook, symmetric with MatMulNBits, and fixed the prepack to always use the SM80 layout (#28749, #28978, #28965)
• Optimized the QMoE SoftmaxTopK router for small-batch decode, added a Split-K2 SwiGLU GEMV kernel, defaulted to fp16 accumulation for fp16 activations, and fused the MoE router bias into the MatMulNBits GEMV (#28980, #29167, #29166, #29170)
• Added a batched small-M GEMV path for 4-bit/8-bit MatMulNBits (#29451)
• Enabled fpA_intB GEMM in CUDA builds with configurable options, added prepacked fpA_intB weight support, and enabled native SM90, block_size=32, and fused bias (#29622, #29499, #29585)
• Fixed a QMoE profiler cross-stream race and made it CUDA-graph-capture safe (#29584)

Coverage, dependencies & fixes

• Added cuDNN-free ArgMax/ArgMin/ReduceSum and fixed LogSoftmax on the plugin EP (#29620)
• Filled the CUDA opset gap for Softplus and Softsign up to opset 22 (#28982)
• Fixed the XQA GroupQueryAttention cudaErrorInvalidValue on Blackwell (sm_120) (#29706)
• Initialized large-head FMHA shared memory per kernel variant (#29140)
• Fixed CUDA/cuDNN DLL preload paths for the CUDA 13 consolidated wheel layout, and fixed the libcudart.so.13 hard dependency that broke import on CPU-only Linux (#29202, #29590)
• Fixed a null allocator passed to plugin EP kernel PrePack, and plugin EP allocator deleter lifetime (#29658, #29663)
• Removed unused code in moe_kernels.cu (#29295)

WebGPU EP

• Added KV cache quantization for ORT WebGPU (#28059)
• Fused FlashAttention decode kernels and extended them to any sequence length (#28389)
• Fused Q/K RMSNorm into GroupQueryAttention for Qwen3-style models (#28484)
• Gemma 4 enablement: indirect dispatch for FlashAttention graph capture, and INT64 support for Equal/Sub/Where/ReduceSum under the enable_int64 flag (#29236, #29392)
• Added a session-level buffer pool for graph capture reuse and made max_num_pending_dispatches configurable (#28761, #28894)
• Added 2-bit quantization support to GatherBlockQuantized (#29054)
• Enabled Cast to int64 by default and switched to naive reduction (#28804, #28174)
• Default non-debug validation mode is now disabled (#28679)
• Tuned parameters for the gemm-subgroup kernel (#28760)
• Enabled FlashAttention for batched GQA with right-padded prompts and fixed batched right-padded prefill with do_rotary (#29247, #29002)
• Fixed a GQA decode split-reduce head_size out-of-bounds race, the past_state == present_state buffer aliasing case, and GatherBlockQuantized dispatch failure for empty indices (#29593, #28753, #29030)
• Fixed nearest half-tie handling for round_prefer_ceil/round_prefer_floor (#28757)
• Skipped MatMulNBitsMlpFusion when the kernel is unavailable (#29089)
• Fixed profiling timestamp alignment with the ORT profiler (#29021)

CoreML EP

• Added Where and And builders (#28597)
• Fixed extended minimal builds with the Core ML EP (#29387)
• Fixed the "model_path" must not be empty error (#29394)
• Fixed CoreML-enabled static builds on macOS (#27960)

QNN EP

• Fixed a segfault in PadNodeGroupSelector::Check when dq_nodes is empty (#28733)
• Added a Gather handler for the transpose optimizer (#28755)

WebNN EP

• Added support for 8-bit MatMulNBits (#29025)
• Removed unnecessary casts around normalization ops (#29024)

OpenVINO / VitisAI / DML / XNNPACK / VSINPU

• OpenVINO EP development updates (#28954)
• VitisAI: recognize native-endian in-memory address tag in process_ext_address (#29248)
• DML: fixed batched readback size (#28680)
• XNNPACK: fixed Clip/Relu fusion when the producer output has other consumers (#28814)
• VSINPU: fixed EP build error (#29440)

CPU & Core Optimizations

MLAS

• Added AVX512 (+VNNI) 2-bit weight CPU kernels (#29064)
• Added ARM64 2-bit weight CPU kernels (#29466)
• Extended the NHWC-sans-transposes path to support depthwise convolutions, and fixed NHWC conv support gating (#28565, #29127)
• Added RISC-V RVV INT8 GEMM/GEMV, M=1 routing, and activation kernels (#28308)
• Fixed a fastmath corner case in sbgemm_neon_kernel and fixed failing KleidiAI NHWC unit tests (#28394, #29010)
• Hinted GCC >= 12 on aarch64 for bf16/fp16 capability probes (#28785)

CPU Attention & Kernels

• Added flash attention for non-quantized CPU GroupQueryAttention, plus a FP32 GEMV decode kernel (#28962, #29216)
• Fixed packed-QKV and broadcast-head bias strides in quantized GQA flash attention (#28963)
• Enabled pre-packed weight sharing for CPU MatMulNBits (#29163)
• Added int8/uint8 CPU support for SpaceToDepth and int8 for DepthToSpace (#29154)
• Fixed CPU Attention causal mask alignment and is_causal bottom-right alignment for external KV cache (#29050, #28958)
• Fixed CPU GQA NaN output for right-padded batched prompts with rotary embeddings (#29069)
• Fixed ReverseSequence returning zeros for zero-length sequences (#28759)
• Fixed a regression in RandomForestClassifier binary classification predictions (#28685)

Graph, Optimizer, and Fusion

• Fixed NHWC second-pass EP assignment cleanup (#27858)
• Rejected QDQ Gemm → QGemm fusion when alpha != 1 with bias, and validated DQ scale/zero-point shapes before QGemm fusion (#28131, #28714)
• Fixed incorrect output for QDQ Resize with non-nearest interpolation modes under ORT_ENABLE_ALL (#28454)
• Fixed INT64 initializer data propagation and the Shape → Gather → TopK rank-1 regression (#28778, #29084)
• Fixed a TransposeOptimizer type error on zero-point-less DequantizeLinear (#29192)
• Fixed SimplifiedLayerNorm fusion with a node-produced Pow exponent (#29196)
• Fixed DropQDQ not dropping Q/DQ around opset-22 MaxPool (#29201)
• Recovered Conv/ConvTranspose rank from weights when the input shape is unknown (#29149)
• Fixed STFT complex input frame offsets (#28961)

Web & JavaScript

• Forwarded WebGPU EP buffer cache mode options from JS (#29017)
• Added a Python implementation for wgsl-gen and removed the dynamic WGSL generator path (#28355, #29141)
• Switched NPM publishing to consume from the CUDA 13 pipeline, and fixed the Node.js packaging stage to use CFS (#28773, #29006)
• Fixed a Dawn parallel build race condition in the wasm_Release / build-wasm CI (#29040)

Language Bindings

Python

• Added an API for HOST_ACCESSIBLE OrtValue allocation (#28038)
• Made the TF converter tolerate a missing requests package (#28825)

Java

• Released outputHandlesArr in the OrtTrainingSession.evalStep JNI binding (#29576)

Rust

• Fixed unbounded lifetime on WithOutputTensor (#29251)

Build, Packaging & CI

• Updated the cpuinfo dependency and integrated cpuinfo_deinitialize() (#28245)
• Built with abseil 20250814 under NVCC (#28586)
• Fixed the Linux CUDA 13.3 build (abseil + CCCL parse errors) and the Windows CUDA 13.3 build for VS 2026 (#29042, #29266)
• Fixed the Windows build for SM90 and later, and CUDA packaging build errors (#29776, #29580)
• Fixed the wrong ORT version in Windows DLL VersionInfo for pipeline builds (#29545)
• Fixed Python and CUDA packaging pipeline failures (#29200, #29241)
• Installed Java in the Windows NuGet packaging job and added Java setup to the Windows x64 QNN CI pipeline (#29774, #29754)
• Sharded Windows ASan test runs and fixed ASan OOM in QDQ Gemm transformer tests (#29503, #28797)
• Propagated and fixed PIP_INDEX_URL piping for container build stages (#29803, #29823)
• Removed deprecated docfx in favor of a newer docs validation method (#29812)
• Fixed a build failure from a direct ONNX schema include in layernorm optimizer tests (#29073)
• Test stability: fixed a data race in CUDA graph tests, a non-deterministic CUDA custom-op test, a thread-pool parameter self-assignment, duplicate schema registration, and incorrect string tensor creation (#29398, #29082, #28799, #29219, #29218)
• Encapsulated the global random seed in a function and improved LinearRegressor test coverage (#28979, #29083)

Contributors

Thanks to our 103 contributors for this release!

@adrastogi, @ankitm3k, @apsonawane, @ArsalanShakil, @bachelor-dou, @bopeng1234, @cbourjau, @chilo-ms, @chunghow-qti, @chwarr, @claude, @Craigacp, @crvineeth97, @dabhattimsft, @daijh, @danielsongmicrosoft, @derdeljan-msft, @edgchen1, @elwhyjay, @ericcraw, @eserscor, @fanchenkong1, @feich-ms, @fs-eire, @FuZoe, @gaugarg-nv, @gblong1, @GopalakrishnanN, @grybouilli, @guschmue, @haoxli, @hariharans29, @hnsyprst, @Honry, @intbf, @ishwar-raut1, @jambayk, @Jaswanth51, @jatinwadhwa921, @javier-intel, @jiafatom, @Jiawei-Shao, @JonathanC-ARM, @justinchuby, @jwludzik, @Kotomi-Du, @kpkbandi, @lucka-me, @maoger, @martin-klacer-arm, @maxwbuckley, @MayureshV1, @mc-nv, @mcollinswisc, @mdvoretc-intel, @mingyueliuh, @mklimenk, @n1harika, @namgyu-youn, @neilmsft, @orlmon01, @Osamaali313, @prathikr, @preetha-intel, @psakhamoori, @qiurui144, @qjia7, @qti-hungjuiw, @qti-yuduo, @quic-muchhsu, @raagrawal, @RajeevSekar, @Reranko05, @Rishi-Dave, @RyanMetcalfeInt8, @Sammy-Dabbas, @sanaa-hamel-microsoft, @sayanshaw24, @selenayang888, @sfatimar, @sgbihu, @Shivani767, @ssam18, @susbhere, @sushraja-msft, @tairenpiao, @the0cp, @tianleiwu, @titaiwangms, @umireon, @vthaniel, @wangw-1991, @wenqinI, @xadupre, @xenova, @xhcao, @xiaofeihan1, @xieofxie, @xuke537, @yen-shi, @yinli-systems, @yuslepukhin, @ZackyLake

Full Changelog: v1.27.1...v1.28.0