Overview
New version has been released.
Nightly build: b5130
More info: dist : releases and versioning of ggml-org projects
Changelog since v1.9.3
927cfce metal : remove leftover ggml-metal.metal kernels file (#4051)
dd80eb2 scripts : use sed instead of grep for version parsing [no ci] (#4052)
1fa6dfa ci : update WoA CUDA 13.4 to use 13.4.1 GA (#4053)
a2b36eb whisper : bump version to 1.9.4 (#4050)
6fb4cd6 ci : add Windows On ARM support to release job (#4048)
c44b60b whisper : call encoder_begin_callback before language auto-detect (#3936)
6d0ed91 whisper : re-seed decoder 0 between calls (#4025)
cec4dbe ci : update close-issue job to not close issues (#4045)
79f2d92 tests : load backends before init when built with GGML_BACKEND_DL (#4031)
61e6cca server : return language in detect response (#4035)
52a939a sync : ggml
a937f4e ggml : bump version to 0.23.0 (ggml/1618)
11d4eec metal : add remaining fa-vec tunings for M3 Max (llama/28373)
140e57a ggml : replace compile definitions with version.h.in (llama/28364)
e2389eb ggml : rename and make private ggml_op_alloc_size_may_expand() (ggml/0)
1b37bea ggml : don't crash when backend search path can't be read (llama/28271)
f32e6fa ggml : remove GGML_CUDA_PEER_MAX_BATCH_SIZE (llama/28177)
e1bbe40 ggml-cpu(s390x) : fix q5_1 uninitialized v_acc (llama/28332)
d1e0e64 sycl: fuse rms_norm+mul+add and add+add residual chains (llama/27610)
36f170e SYCL: Refactor GGML_SYCL_ENABLE_MKL_FA to global var (llama/26863)
d784add opencl: quant lm_head / decode GEMV and medium-batch GEMM optimizations (speculative decoding/MTP) (llama/26477)
0a4a95c tune MMVQ to MMQ crossover for SM87 (llama/28285)
4dd48dd metal : add sparse FA (llama/28098)
d55d345 metal : fix glu dispatch with ne00 = 1 (llama/28306)
25350b5 CUDA: Allow concurrent streams per split for multi-GPU (llama/28198)
47d348a vulkan: fix FA dequant path engagement (llama/28190)
f24a386 sycl : enhance the api to support peer-to-peer copy (llama/27550)
a704770 sycl: reduce redundant work in Q4_K multi-column MMVQ (llama/27062)
e560569 finetune: fix no KV cache (llama/27199)
37f0f44 ggml-hexagon: add F16 support for unary ops (llama/28228)
1bdda1e metal : add fa-vec tunings for M3 (llama/28236)
3a1c7d6 metal : fix memory query under low-memory conditions (llama/27701)
4d343d7 ggml-cuda : remove unused vars (llama/28235)
519df61 CUDA + ggml: add sparse-fa for DSV4/GLM (llama/27970)
c2b4007 ggml: avoid KleidiAI buffer type init on dispatch (llama/27891)
dc70853 hexagon: MUL_MAT and MUL_MAT_ID fusion and fixes (llama/28202)
1c7d35e vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec (llama/27449)
a9e5861 vulkan : only request VK_KHR_shader_bfloat16 extension if supported (llama/28155)
d57ae98 ggml-cpu : conditionally add SpacemiT IME kernel sources (llama/27961)
35133c9 opencl: fix out‐of‐bound reads in the Adreno image kernels (#27632)
c94921f hexagon: add missing FARF logs for cpy/get_rows/set_rows/gdn ops (llama/28217)
fcc2fee metal : add metallib build support for xcframework (llama/28163)
2c48678 cuda: fuse MoE weighted expert reduction (llama/25952)
f162a19 Revert "sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 12…" (#28184)
408faaa sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 1280 (llama/28016)
5f07f85 metal : add fa-vec tuning for M2 Pro (llama/28122)
8cca1a3 metal : add fa-vec tunings for A18 Pro (MacBook Neo) (llama/28152)
a245a8f metal : fix more leaks due to missing autoreleasepools (llama/27883)
870db2a metal : add fa-vec tuning for M2 Max (llama/28015)
4f3a2a4 sycl : support limit max alloc memory within 2GB for host-pinned memory (llama/27559)
5032008 metal: enable Metal 4.0 tensor API on M5+/A19+ (llama/27461)
8e54c65 metal : add fa-vec tunings for M1 Ultra (llama/28088)
f22bb2e CUDA: XOR swizzle flash attn K,V smem fp16 tiles (llama/25635)
dbc40ef metal : add concat support for quantized types (llama/28116)
2f608ab AVX2: Speed up large batch size prompt processing of IQ models (llama/27402)
c6934d0 metal : add top-k radix implementation (llama/28073)
088c603 opencl: tune the quant paths for Intel Xe-LP GPUs to improve its TG and PP performance (llama/26438)
c648b9a webgpu : avoid crash when offset is not multiple of 4 in WebGPU ggml_backend_tensor_get() implementation (llama/28045)
c1be45b ROCm: add radix TOP_K for long rows (llama/27466)
7614a4c metal : add fa-vec tunings for M1 (llama/28078)
b0f4bc0 CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were restricted to 1 token (llama/27621)
76a51e8 sycl : Enhance to get the free memory of Intel GPU (llama/27968)
6ce7b89 vulkan: tune mat-vec rows for batched inference on Strix Halo (llama/27909)
96dddd8 ggml : add MUL_MAT to the list of ops that may need additional memory (for WebGPU) (llama/28071)
db00b01 vulkan: top_k radix select for k >= 1024 for Qwen 3.8 Flash Next (llama/28032)
01ebd22 hexagon: fix CPY fence bug (llama/28033)
e5c96ca metal : add remaining Q4_1/Q5_0/Q5_1 fa-vec tunings for M2 (llama/28017)
4089fa6 rpc: avoid serializing buffers from other servers (llama/26500)
749683d ggml : fix ggml_backend_buft_get_alloc_size() guard (llama/28038)
e9583f0 ggml: add SWIGLU_CLAMP (llama/27930)
e900a73 CUDA: use the fast mm_ids_helper path for any n_expert_used (llama/27978)
35d9e22 hip: tune rdna 3 mmq config (llama/26284)
e5c9e3e hip : optimize Q2_0 dot-product path for gfx1201 (llama/26753)
4b2243a ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (llama/27960)
43acf3d rpc: fix apple rdma error spew on teardown (llama/27908)
1e0f382 metal: add fa-vec tunings for M3 Ultra (llama/27999)
b66593e metal : Add fa-vec tuning for M3 Pro (llama/27963)
5e49459 rpc : fix pre-rdma macOS versions (llama/27815)
3ad8b9b hexagon: support for device discovery and create sessions on demand (llama/27785)
3d4e0e9 sycl: split long rows in TOP_K instead of one work-group per row (llama/27847)
c68f205 metal : fix null-pipeline crash for F16 src1 mul_mat/mul_mat_id (llama/25648)
c969c68 ggml: allow passing alloc dependencies in graph_optimize (llama/27301)
b33bbc5 metal : add fa-vec tunings for M2 (llama/27940)
2a11026 opencl: use a better matmul path on two Adreno GPU generations (llama/27640)
285f1ff metal : assert shared memory padding (llama/27951)
308fa4f metal : add remaining fa-vec tunings for M4 Pro (llama/27915)
325c8d1 sycl: make --fit respect --fit-target better (llama/27629)
e644752 vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs (llama/27526)
590fe18 metal : add fa-vec tunings for M1 Max (llama/27932)
d501a0a vulkan: Change mul_mat_id to pad K rather than N (llama/27925)
4c38040 vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize (llama/27812)
caea96f ggml : fix conv_transpose_2d for multiple batches (llama/26132)
ba99c09 Vulkan: add hoisting support for row IDs and expert count in shaders (llama/26686)
0a15087 metal : add fa-vec tunings for M4 (llama/27875)
7f78e1b OpenVINO: Update OV to 2026.3.1, whisper.cpp support, Qwen3.5 on NPU, and new ops (llama/27843)
fa4d244 sycl: use TILE for quantized KV decode on BMG (llama/26689)
97d0da2 sycl: bind the f16 KV cache in place for the oneDNN SDPA path (llama/27468)
530e3f4 metal : add fa-vec tunings for M3 Max, M5 and M5 Pro (llama/27863)
ff38b98 metal : add fa-vec tunings for M4 Pro (llama/27824)
b6571e4 ggml-hexagon: add HTP unary ops for ABS and LOG (llama/27786)
a0614d9 hex-unary: fix RMS_NORM_MUL weight-offset bugs for grouped/broadcast norms (llama/27798)
8529971 opencl: add bin kernels kernel_gemm_moe_q4_0_q8_1_dp4a_bin, kernel_gemm_moe_mxfp4_q8_1_dp4a_bin (llama/27768)
55ab1e5 Feature: Added LIGHTNING_INDEXER support for Deepseek V4 ops on Vulkan Backend (llama/27453)
a5db1d6 metal : fix memory leaks due to missing autoreleasepools (llama/27758)
3fea10d hexagon: support for multi-NPU devices (IQ9, IQ10) and fully asynchronous backend (llama/26501)
5271734 vulkan: warptiles currently assume warp sizes <= 64, clamp to work around larger warps (llama/27726)
0a02697 Implemented vulkan cross_entropy_loss and cross_entropy_loss_back (llama/27216)
82f5f85 rpc : implement event and async backend APIs (llama/18626)
9d8e6b9 cuda: unblock mmq for MoE on sm_60 (llama/26264)
8c0adb0 ggml-metal: add chunked SSD MMA for Mamba-2 prefill optimization (llama/26647)
8df657a ggml-meta: propagate buffer usage and call init on the new tensors (llama/27586)
482956e kleidiai: Rework KleidiAI Build System/Integration (llama/26077)
e820c28 rpc: support apple RDMA as an RPC transport (llama/26421)
be12d39 metal : null-check buffer alloc to fix OOM crash (llama/25371)
642b5d3 ruby : Add #free method, check MemoryView strictly (#4032)
eacbd82 whisper : default-initialize whisper_mel to avoid uninitialized read (#3981)
c4ac001 parakeet : fix TDT decode by outputting raw logits from the joint graph (#4017)
9781133 talk-llama : sync llama.cpp
3680f66 pi : init
0414519 sync : ggml
d470c9d ggml : bump version to 0.22.0 (ggml/1607)
322a77c sycl : mark tq2_0 as not supported (llama/27660)
17a522a webgpu : fix handling of infinity values during ARGSORT and TOP_K (llama/27538)
fa3b87c metal : per-device tuned (Q, NE) for flash-attn vec (llama/26570)
fe52277 sync : ggml
b15d31d metal: per-op source split + parallel compile (llama/26561)
4257445 scripts : update ggml-am
1d8e052 sync : ggml
aa25d33 ggml : shorten virtual device naming in CUDA and Metal (llama/27608)
103305e webgpu : reorder includes since V that appears in common_decls.tmpl may be defined as K in flash_attn_decls.tmpl if KV_OVERLAP (llama/27545)
32d0f49 ggml : fix ggml_clamp (llama/27644)
20209c2 Deepseek 4: -sm tensor (llama/26490)
3b89b37 Fix meta tensor split state propagation (llama/27574)
c8a4009 cuda : add POOL_1D support (llama/27573)
13a7856 vulkan : added the PAD_REFLECT_1D operation (llama/26586)
1efb31e ggml: optimize concat op by replacing per-element memcpy with row-level memcpy (llama/24575)
21a67dd sycl : add Q2_K reordered MMVQ and ESIMD kernels (again) (llama/27490)
5f7bd9d opencl: fold the gpt-oss MoE per-expert bias adds into the epilogue (op/kernel fusion) (llama/26431)
a722846 whisper : guard null source in buffer loader read callback (#3982)
c122757 docs : center badges in README.md [no ci] (#4012)
2569409 devops : add main-rocm Dockerfile (#3975)
52dec9d vitisai : add VitisAI Plugin for AMD Ryzen AI NPU encoder offload (#3608)
233fe1f whisper : bypass cross-attention scaling for OpenVINO backend (#3997)
51de5e8 openvino : update model conversion and README.md (#4003)
3391d6b scripts : add release.sh script (#4010)
a4610c7 docs : add release badge and remove stable/roadmap [no ci] (#4009)
ab57887 make : add --parallel to cmake build command (#4007)
45f1593 sync : ggml
ce77728 Revert "sycl : add Q2_K reordered MMVQ and ESIMD kernels (llama/26336)" (llama/27486)
0d9ba28 ggml : bump version to 0.21.0 (ggml/1597)
d6c416e kleidiai : add SME2 F32 GEMV kernel support (llama/26891)
d60ef65 sycl : add Q2_K reordered MMVQ and ESIMD kernels (llama/26336)
b1cb805 sycl : Add Q5_K ESIMD kernel (llama/26376)
2cb52dd opencl: keep the vocab-scale K-quant lm_head on the CPU for Adreno A7X (compiler issue workaround) (llama/26440)
af74f97 sycl: Update gate logic for Alchemist GPUs regarding OneDNN features. (llama/26635)
f19250e sycl: fix multiple warnings in compiling sycl backend (llama/26713)
12137c3 sycl : fix load model with mlock issue (llama/27250)
5656e44 ggml: support ggml_rope_set_offset on opencl, sycl, wgpu, hexagon (llama/27345)
d68216a metal : clamp K extent in tensor API mat-mat kernel for K not a multiple of 32 (llama/27450)
c16cb42 opencl: fix q6_K flat mul_mat for Adreno A6x/A7x GPUs with older E031 compilers (llama/26476)
73c2b7e opencl: fix local size for norm (llama/27339)
8539d72 vulkan: FA MMQ should use fp32 for Q quantization calculations (llama/27413)
60f14a5 metal : dequant kv cache only for large batches (llama/27438)
2922418 CI: Use LLVM's OpenMP over MSVC_DEBUG_non_redist on Windows (llama/26678)
c2dc946 CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover (llama/26079)
283775e metal : dequantize quantized KV to F16 before flash attention (llama/27390)
02be8f5 Revert "tensor-split meta backend fixes (ggml/26502)" (llama/27433)
13fa860 ggml: fix backend split scheduler race condition (llama/26040)
2648a70 ggml-cuda: provide static workspace for cuBLAS handles (llama/26574)
acfad32 vulkan : add source groups for shaders (llama/26666)
a3386d4 opencl: make the MoE expert scatter deterministic (llama/26464)
135f348 tensor-split meta backend fixes (llama/26502)
9f4b18a hexagon: fix FA HMX queue ordering and pack the rescale D matrices (llama/27042)
cd340ea opencl: port fused ssm_scan kernel (Mamba-2, d_state in {128, 256}) to GPU (llama/26439)
3d27742 ggml-cpu: gate __fp16 on __ARM_FP16_FORMAT_IEEE (llama/26860)
11e58f3 vulkan : dequant q8_0 KV once in coopmat1 (llama/25494)
4ef3e87 vulkan: add null checks in ggml_vk_queue_command_pools_cleanup (llama/27353)
689ad69 sycl: report zero devices instead of aborting when the host has none (llama/27291)
8442c74 ggml: add ggml_rope_set_offset (+ metal support) (llama/27120)
d830bd2 metal : dequantize q8_0 using packed types (llama/27370)
1c882a8 vulkan: tiled transpose for 0<->2 permuted CONT (llama/26585)
7df5fa8 ggml-webgpu: add mulmat with overlapping src0/src1 (e.g., for minimax-01) (llama/27321)
fa1e2bc opencl: fix WAR race in the generic FA tile kernels when the WG spans subgroups (llama/26434)
4be3101 RPC: populate use_count to enable fusion inside backends (llama/27142)
dbab353 sycl: honor GGML_HINT_SRC0_IS_HADAMARD (llama/27298)
c6a2bd0 devops : stop shadowing proper cuda libraries in runtime docker build (#3989)
ec73735 tests: add check for nullptr for wctx in test-vad-full (#3998)
a4ad15f ci : release clean-up (#4005)
81a3fad server : only enable token timestamps when the response needs them (#3990)
d61695d docs : fix typos in readme files (#4004)
b78df3d ci : move close-issue.yml to .github/workflows [no ci] (#4002)
339f2b4 bindings-javascript : remove package.json from git (#4001)