Highlights

GLM-5.2 NVFP4, tuned for production: We took time this cycle to tune GLM-5.2 NVFP4 on Blackwell for optimized production serving. It now runs at 500+ tok/s/user on 8x B300, 450 on 4x GB300 (bs=1). Run GLM-5.2 with our cookbook.

Spec V2 by default: zero-overhead scheduling via CUDA-graphable DSA draft-extend, dropped D2H/H2D syncs, fused metadata ops. +11% end-to-end TPS (#29413, #29343, #29166, #29077).
IndexShare MTP: reuses the indexer top-k across draft steps, up to 1.9x lower draft-step cost at long context (#29959, #29787, #29654).
TopK V2: fuses top-k selection with the page-table transform, runtime k up to 2048 (#30274, #26788).
Indexer prologue fusion: 12 kernels to 4, ~8% faster decode at bs=1 (#27705).
GEMM: shape-specialized JIT router GEMM + CuteDSL BF16 GEMM for Blackwell (#21531, #30117).
FlashInfer autotune: now covers draft-model graphs (#29595).

New Model Support: Hunyuan 3 (Hy3), Hierarchical Reasoning Model (HRM-Text), NVIDIA LocateAnything-3B, Baidu Unlimited-OCR, JoyEcho multi-shot A/V, plus Qwen3.6 NVFP4 support.

Native web search (Exa): Built-in web_search support backed by Exa (#29342).

Breakable CUDA Graph on by default: Breakable CUDA Graph is now the default capture path, reducing per-step kernel-launch overhead (#29458); full CUDA Graph support for the prefill phase lands as experimental (#27988).

Linear-attention kernels (KDA / GDN): New FlashKDA prefill backend for safe-gate KDA linear attention (#29472), plus ReplaySSM buffered output-only decode for linear attention (#28451).

FlashInfer A2A for routed MoE: Adds FlashInfer all-to-all with the flashinfer_trtllm_routed MoE runner (#22394).

DeepSeek-V4 Optimization:

Optimizes C128 state-pool allocation using the request state pool (#28612).
FlashMLA sparse prefill is now enabled by default for DeepSeek-V4, reaching >10% throghput gain on long context. (#29775).
Non paged indexer support for long context prefill, with >5% e2e throughput gain.(#29619).

Decode Context Parallelism: decode context parallelism lands for MLA models, including DeepSeek V3 and Kimi K2 series (#14194).

Dependency upgrades: transformers bumped to 5.12.1 (#29393); tvm-ffi / sgl-deep-gemm / tilelang upgraded (#29554).
and . See the DeepSeek-V4 cookbook.

Full release notes by category below.

New Model Support

Hunyuan 3 (Hy3): #30201 (cookbook)
Hierarchical Reasoning Model (HRM-Text): #27887
NVIDIA LocateAnything-3B: #28958
Baidu Unlimited-OCR: #29186 (cookbook)
JoyEcho multi-shot A/V: #27420 (cookbook)
Qwen3.6 ModelOpt mixed NVFP4: #27906, #29905 (cookbook) ⭐

GLM-5.2

[BCG][GLM5] perf: BCG support and prefill enhancements: #27053 ⭐
[CI] Add GLM52 NVFP4 MTP B200 tests: #30021
[cookbook] GLM-5.2 NVFP4 B300: TP8 recipe + 3 strategies: #29557
[Cookbook] GLM-5.2: tune GB300 NVFP4 recipes + fill benchmarks: #29486
[cookbook] drop redundant serve flags (GLM-5.2) + fix M3 page-size note: #28731
[Docs] Add NVFP4 quantization to GLM-5.2 cookbook: #29380
[DSA][GLM5.2] Index Share for MHA: #29959
[GLM-5] Tune the threshold of router GEMM: #29470
[Spec] Anchor GLM-5.2 MTP IndexShare topk on the draft-extend step: #29787
Bypass legacy GLM DSA layer types validation: #29454
docs: add B200 NVFP4 recipes + benchmarks to GLM-5.2 cookbook: #29674
docs: add PD disaggregation to GLM-5.2 cookbook playground: #29544
docs(cookbook): add AMD MI300X/MI325X/MI355X support for GLM-5.2: #28471
Fuse the DSA (V3.2, GLM-5.x) indexer Q/K paths into single kernels: #27705 ⭐
glm5.2 on ascend doc (new version): #29828
Support JIT fused A GEMM (MLA down projection) and support GLM-5 hidden size, SM120: #27397
Update GLM-5.2 B300 and GB300 NVFP4 cookbook settings: #29466
Update GLM tests to 5.2 and delete redundant tests: #29686
[AMD] [GLM5] Guard cuda_runtime.h for ROCm in fused_metadata_copy: #29373
[AMD] [GLM5] Mark EAGLE verified on MI300X/MI325X (gfx942) in GLM-5.1 cookbook: #29313
[AMD] [GLM5] GLM-5.1 MXFP4 (MI355X) + enable EAGLE for gfx950 in cookbook: #29194
[AMD] [GLM5] Add opt-in Triton fp8 sparse-MLA prefill kernel for gfx950: #28975
[AMD] [GLM5] skip redundant -inf pre-fill of HIP indexer MQA-logits: #28757

DeepSeek V4

[DeepSeek V4] Enable FlashMLA sparse prefill by default: #29775 ⭐
[DeepSeek-V4] Add an opt-in non-paged indexer for long-context prefill: #29619 ⭐
[DSA] Fold page-table into fused top-k v2 (decode): drop page_size=1 expansion: #30274
[JIT Kernel] DeepSeek-V4 DSA indexer: faster top-k + page-table transform (runtime k <= 2048): #26788 ⭐
[Cherry-pick to release/v0.5.15] [DSA] Fix IMA in fused top-k v2: write all output slots on tie overflow (#30512): #30559
[Cherry-pick to release/v0.5.15] [DSV4] perf: Make FP8 quant output tensor contiguous (#27926): #30449
[Cherry-pick to release/v0.5.15] [DeepSeek-V4] Enable non-paged indexer by default for large prefill chunks (#30140): #30436
[Cherry-pick to release/v0.5.15] [DSA] Re-enable fused top-k v2 for MTP: clamp padded-row seq_lens to >= 0 (#30378): #30427
[AMD] Improve performance of DSV4 in high concurrency: #28938
[AMD] DSV4 aiter reduce-scatter decode: #29103
[AMD][DSV4] Remove per-batch D2H syncs in MTP to avoid bubbles between 2 batches: #29420
[AMD][DeepSeek V4] Fix default FlashMLA sparse prefill off on ROCm/HIP: #29982
[AMD] Fix DeepSeek V4 MTP accuracy issue: #30333
[AMD] Fix dsv4 indexer dtype dispatch on gfx950: #29479
[AMD] Cap DSV4 Flash max_total_num_tokens: #30313
[AMD] Fix DeepSeekV4 server cutlass error: #30374
[Intel GPU] DeepSeek V4 10/N : Add sqrtsoftplus support to fused_topk_torch_native: #28048
[Intel GPU] DeepSeek V4 7/N: Support fused_rope_inplace on XPU using triton: #27915
[Intel GPU] DeepSeek V4 3/N: Support hc_split_sinkhorn on XPU using sgl_kernel: #27783

Speculative Decoding

[MoE] Fix moe_fused_gate out-of-range expert id on all-NaN rows (fixes eagle_dp_attention crash): #30079
[perf] tiny optimize select_index op for draft extend: #29078
[perf] simplify _apply_cuda_graph_metadata for draft extend in trtllm_mla backend: #29077
[Spec] Enable FlashInfer autotune for spec draft: #29595
Speculative decoding support on XPU: #23180
(perf): Shard Kimi-K2.5 Eagle3 draft fc + symm-mem AG: #29223
[XPU] Unbreak stage-b: re-add --disable-decode-cuda-graph, quarantine EAGLE3 parity: #30048

Piecewise & Breakable CUDA Graph

[Experimental] Full Cuda Graph Support for Prefill: #27988 ⭐
Disable dsr1 prefill cudagraphs by default: #28053
Enable Breakable Cuda Graph as Default: #29458 ⭐
[AMD] Enable BCG on ROCm + route aiter prefill via MHA during PCG/BCG capture for Kimi-2.5: #27833
[XPU] Enable XPU graph support (decode full-graph + prefill tc_piecewise): #29053

Attention Backends

[GDN][KDA] ReplaySSM buffered output-only decode for Linear Attention: #28451 ⭐
[KDA] Add FlashKDA prefill backend for safe-gate KDA linear attention: #29472 ⭐
[KDA-Pilot] Add LTX2 QKNorm split-RoPE CUDA fast path: #29708
[KDA-Pilot] Add diffusion residual-gate CUDA fast path for LTX2: #29361
[KDA-Pilot] Add diffusion causal Conv3D cat-pad CUDA fast path for Cosmos3: #29281
[MoE] Consolidate ungrouped + grouped gate/topk onto one Triton router (#26771) — faster than AOT on B200/H100/H200, at parity with flashinfer: #29771
[NVIDIA] Support flashinfer a2a with flashinfer_trtllm_routed moe: #22394 ⭐
Re-enable SM90 FlashInfer allreduce fusion with safe backend defaults: #28789
[CPU] optimize GDN prefill performance: #29117
[NPU] perf: precompute mamba conv-state track indices once per batch: #29105

MoE & Expert Parallelism

[JIT Kernel] Triton moe fused gate: #25835
[MoE] Retire the AOT moe_fused_gate / kimi_k2_moe_fused_gate gate kernels (#26771): #29997
[Perf][Kernel] Fuse SiLU+Mul into NVFP4 Expert Quantization for CUTLASS MoE: #18612
Add GB10 FP8 fused MoE Triton config: #25665
[AMD] Fuse shared-expert append + DeepEP remap into one Triton kernel: #28450
[AMD] fix(moe): correct fused shared-expert scaling on aiter/DeepEP path (mori all-to-all): #28237
[AMD] Implement QuarkW4A8MXFp4MoE to support amd/gpt-oss-120b-w-mxfp4-a-fp8: #27204
[AMD][MORI-EP] Skip LocalExpertCount kernel in decode graph when not recording: #30302
[NPU] [DOC] add missing DEEP_NORMAL_MODE_USE_INT8_QUANT for w8a8+deepep scenarios: #29937

Quantization

[diffusion] Add Qwen-Image ModelOpt NVFP4 support: #28928
[Kernel] Add SM90 Q8KV8 FP8 Sparse MLA Prefill JIT Kernel with Tests and Benchmark: #25751
[weight checker] refactor: add precision branch; allow ULP quant err; used chunked compare: #28974
Add Intel Quantization Support in SGLang: #18139
Fix gfx95 bpreshuffle FP8 activation scale layout: #29275
✨ [llm][npu][quant] Add W4A8 MXFP quantization support for Qwen3 Dense on Ascend NPU: #23650
[AMD] Gate broken CK block-FP8 GEMM shapes to aiter-triton-GEMM to fix ROCm 7.0 Qwen3.5 accuracy: #29918

Parallelism & Disaggregation

[feature] implement dcp for deepseek_v2: #14194 ⭐
[HiCache] Add NIXL FILE cache cleaner: #28258
hisparse: support NIXL DRAM KV destinations for HiSparse: #27563
[AMD]: Enable NIXL PD disaggregation for ROCm(1/n): #28348
[AMD]: docker(rocm) bump Mooncake to latest main + enable multi-protocol: #27730
[AMD] Support triton backend decode context parallel for Qwen3.5: #25090
[AMD] Fix AITER custom all-gather CUDA-graph capture crash under torch_memory_saver: #30557

Scheduler & Runtime

[Perf] Overlap result D2H copy with the next forward step: #29075
[refactor] Add a read-through server_args accessor to RuntimeContext (stack 1/15): #30063
[scheduler] Add scheduler metrics reporter init hook: #29535
Add native Exa-backed web_search support: #29342 ⭐
Add scheduler metrics extension hooks: #29207
feat(metrics): add Prometheus metrics for the EPD encoder server: #27564
Support DP-aware PD router dispatch: #26245

HiCache & Radix Cache

[HiCache] Optimize HiCache hash generation with bulk token byte conversion: #28287
[HiCache][AMD] Add UMBP tiered DRAM + SSD L3 storage backend with hugepage host allocator: #25377
[optimize] fix swa eviction boundary for unfinished inserts: #29350
Optimize C128 state pool allocation using request state pool: #28612 ⭐

Multimodal

[EPD] Optimize multimodal global cache with paged embedding pool: #28441
[VLM] Qwen3-VL / Moss-VL ViT preprocessing optimizations: #28940 ⭐
Add MiMo V2.5 Blackwell vision FA4 recipe: #29253

Model Support & Optimizations

[NVIDIA] Support TF32 matmul to improve MiniMax gate gemm performance: #22744
[trtllm_mha] Fuse cuda-graph metadata rebuild into one triton kernel: #29843
Add fused EH norm for DeepSeek NextN: #29667
Add Laguna XS.2.1 DFlash support to SGLang: #29446
Fused QK GemmaRMSNorm + RoPE + gate kernel for Qwen3.5: #28320
perf(triton): avoid per-step D2H .item() sync in cuda-graph loc translate: #29921
Support Cutedsl BF16 GEMM JIT kernel: #30117

SGLang-Diffusion

[diffusion] perf: add unified SP shard helpers and zero-copy tail-pad attention: #30107
[diffusion] perf: tp-shard every text/image encoder across the full DiT replica (any parallelism): #30086
[diffusion] feat: add LingBot realtime prompt, KV window, and lazy VAE controls: #30040
[diffusion] feat: performance_mode=speed enables torch.compile by default: #30016
[diffusion] feat: add --offload-during-compile to fit max-autotune on tight-memory GPUs: #29862
[diffusion][cache-dit] support Krea-2 + run-driven has_separate_cfg: #29688
[diffusion] feat: support cache-dit for Ideogram 4: #29631
[Diffusion] Add Krea 2 support: #29052
[diffusion] fix: add profiling support and fix VBench dataset handling in bench_offline_throughput: #27704
[diffusion] rl: add sleep/wake support for diffusion engine: #22659
[AMD][diffusion] Disable layernorm torch.compile decorator in eager mode on ROCm to avoid memory-access fault: #29673

AMD / ROCm

[mori] Add a combine-kwargs hook and use_external_inp_buf plumbing: #29097
[AMD] Fuse topk padded-token masking into a single Triton kernel: #28084

NPU / Ascend

[NPU] [DOC] Update arguments detail to NPU support features page: #30328
[NPU] [DOC] Update deterministic inference feature support status to A2, A3: #29632
[NPU]GLM-4.7-Flash optimize with fused kernels: #29509
[NPU] [DOC] Add environment prerequisites to model tutorials: #29293
[NPU] Support fsdp for rl_on_policy_target: #29128
[NPU] adapt_fused_rope_qk_mqa_optimize: #28872

CPU / Intel / XPU

[qwen3.5][XPU]Add XPU support for set_embed_and_head and fused QK RMSNorm kernel: #27870
[Apple Silicon] Add labeler config: #29908
[CPU] add fused_qk_gemma_norm and refactor norm kernel implementation: #30216
[CPU] enable fused_sigmoid_mul on CPU device: #29378
[Intel GPU] add pytorch profiling support for XPU in bench offline throughput and enhance num steps: #28308
[XPU] Remove redundant xpu graph backend and make xpu graph opt-in by default: #29911

Dependencies

[Deps] Bump transformers to 5.12.1: #29393 ⭐
feat(sgl-kernel): add InfLLM v2 attention kernels: #29383
Upgrading tvm-ffi/sgl-deep-gemm/tilelang: #29554 ⭐

All PRs included in this release: v0.5.14...v0.5.15

New Contributors

Approximate list from git history; GitHub's "Generate release notes" produces the canonical, deduped list.

Augusto Yao made their first contribution in #14194
Wang, Mengni made their first contribution in #18139
meinie made their first contribution in #21531
ANSHUMAN TRIPATHY made their first contribution in #23180
Rohit Harkhani made their first contribution in #25071
TzZtzt made their first contribution in #25153
Daniel Stokes made their first contribution in #26255
Jiajun Li made their first contribution in #26980
Kaixi made their first contribution in #27053
Stanley Winata made their first contribution in #27204
hhhh1252023 made their first contribution in #27433
Oxana Korzh made their first contribution in #27835
yifei wu made their first contribution in #27887
chengcuiping made their first contribution in #28401
Raiden Makoto made their first contribution in #28455
Martin Hua made their first contribution in #28481
Tai An made their first contribution in #28503
a60124901 made their first contribution in #28586
Zhihao Wang made their first contribution in #28676
toufupi made their first contribution in #28770
Yuankai Chen made their first contribution in #28787
Anusha Pant made their first contribution in #28952
Jyothirmai Kottu made their first contribution in #28958
qyb233 made their first contribution in #28980
Alex Tumanov made their first contribution in #28996
hirakunaramuka2 made their first contribution in #29004
IvanShan177 made their first contribution in #29102
Shijin Zhang made their first contribution in #29161
Aditya Kamat made their first contribution in #29186
jonah-berman made their first contribution in #29342
Feng Yao made their first contribution in #29350
Feng Yao made their first contribution in #29352
cauphe made their first contribution in #29383
Siming Deng made their first contribution in #29440
adamkbaranowski made their first contribution in #29446
bef0rewind made their first contribution in #29570
SSSunzt made their first contribution in #29571
Jzz1943 made their first contribution in #29631
sushil Dubey made their first contribution in #29672
YukioZzz made their first contribution in #29756
san-tian made their first contribution in #29798
Pranjal Shankhdhar made their first contribution in #29843
NOOB made their first contribution in #30181

Full Changelog: v0.5.14...v0.5.15

sgl-project/sglang v0.5.15 on GitHub