Overview
New version has been released.
Note
More info at 3PM CET
Assets
Nightly build: b12359
More info
Changelog since v0.9.0
b1b770d examples : remove explict version from test-cmake (#32191)
4cda622 llama.cpp : bump version to 1.0.0 (#32190)
6859813 sync : ggml
3bfde28 ggml : bump version to 1.0.0 (ggml/1690)
28d37fe examples : clean-up old/obsolete stuff (#32184)
bd2767d ci: fix HIP quality check by ignoring the FP8 vec FA kernel spill (#32183)
82503b4 ci : set windows-cuda ccache size to 1G (#32185)
5676dfb chat : accept </ifm|think> as K2 Horizon closer at medium/low effort (#30335)
0a66ac7 server, fix CI: read a small temp file in the read_file tool test [no release] (#32168)
892b40b server: allow private media URLs in the vision tests [no ci] (#32167)
e970101 server: skip LAN address tests when the host cannot reach it [no ci] (#32166)
92d6c8a fix(hexagon): make host build work with clang-cl on Windows ARM64 (#32011)
8ca5a80 llama : prune (#30565)
29dbbad ggml : prune (#29801)
2a4daf0 hexagon: add Q5_0 and Q5_1 support (#32094)
d8c8cc4 server : cancel MCP tool calls on client disconnect or timeout (#32025)
3ba1ce0 qwen3.5: merge tensors on convert also (#32102)
0fb8f15 Title: CUDA: few-row MMQ for BS 2-24 (#31533)
827466b ci : build ui on non-slim runner (#32138)
8954f19 server: fix SSRF for CORS proxy and media URLs (#32115)
a1a68eb server: explicit opt-in tools via non-loopback addr, check cors host port (#32113)
e182fda server: disallow direct access child instance from browser (#32120)
09452cd server: keep the server awake while a request uses its state (#32133)
cfb9d4b mtmd: require an F32 position embedding for HunyuanVL (#32131)
72f4603 mtmd: read the MiniCPM-V projection width from ne[1] (#32132)
8c38836 mtmd: require the MLP projector and patch embedding tensors (#32129)
08f4959 mtmd: require a 2x2 spatial merge for the Qwen-VL projectors (#32130)
a8575a3 ggml : route GATED_DELTA_NET to the SpacemiT RVV kernel (#31897)
a963f65 vulkan : Intel FA kernel for prefill. (#29357)
ab4a77b ggml-cpu: use the 4x4 simd_gemm tile on ARM NEON (#32128)
0d69255 CUDA: Optimize prefil via fuse of w_s scale in epilogue MMQ for nvfp4 checkpoints (#26048)
3bce8bd ggml-backend : prefetch only inputs computed by earlier splits (#32100)
81f21f4 ggml : dispatch SpacemiT Q4_1 32x256 through the HP kernel (#31929)
b19f1d9 ui: add search_chats tool (#32051)
d90cfd4 ui: install llama-brand from a GitHub tarball instead of git (#32112)
60d46a5 server: when --tools-runtime is specified, ignore x-tool-runtime (#32119)
93cf736 chat : refactor gemma4 to build loose schema rules and clean up (#31915)
4fffab6 llama : avoid touching unused graph metadata pages (#32043)
09338b8 sycl: make oneDNN FA cache thread-safe and memoize runtime failures (#31138)
dcbff21 sycl : fix AOT for more than 4GB (#31873)
f3d7725 sycl : use cached device-id in set_main_device (#32013)
7021f7e CUDA: add cuda debug buffer pool (#31919)
e0949d0 opencl: add bin kernel kernel_gemm_noshuffle_q4_0_f32_32b_trans_ila_a7_bin (#30947)
5d4c8ee CUDA: skip empty MoE tiles in MMQ stream-k fixup (#32067)
dd6b945 server: use the moved test model in the router CORS test (#32117)
e1f7593 opencl: add decomposed FA for Adreno (#32046)
7bcbc04 ui: move js sandbox setting out of "developer" (#32054)
dd00306 docs: update image link in README.md (#32038)
1d81992 ui: keep downloads in Discover during search and fix their progress with a draft (#31467)
bd717e0 ui: take the logo and favicons from the llama-brand package (#32039)
2cbe0ec server: fix --cors-origins handling + protect against DNS rebinding (#32098)
664f031 ci: move test models to ggml-org-dev (#32096)
4180722 metal : prevent out-of-bounds reads in SOLVE_TRI kernel (#32103)
055c3c2 vulkan: support batched tensor copy API (#31418)
2656b71 vendor : update LibreSSL to 4.4.1 [no ci] (#32106)
cb44fdb llama : reuse graph result instances across scheduler reserves (#32089)
cd2db80 jinja : support chained comparisons and argument unpacking (#32072)
f7b886f chat: allow TTS model to provide custom text normalizer via jinja (#32053)
956591f ci : remove dead snakes (#32084)
ccd360f llama : remove non-fused GDN and ubatch-dependent graph nodes (#32078)
b326dd6 devops : add missing MKL runtime DLLs to the Windows SYCL release package (#32005)
b8abe41 vendor: update cpp-httplib to 0.66.0 (#32049)
2e325dc backend : re-instantiate CUDA graphs when the cluster dims of a kernel change (#31284)
1648d64 test-fp8.cpp: use only the public ggml API (#32083)
4324336 ROCm/HIP: fixed BLAS workspace instead of rocBLAS allocating one per call (#31838)
71a6a11 quantize: fix imatrix GGUF filepath metadata leak (#32000)
7f24aac cpu: accumulate flash attention V in F32 (#29810)
af6b524 webgpu: add PAD_REFLECT_1D support (#31877)
d2b3eb9 vulkan: allow dequantized KV optimization with CM2 (#31526)
eeb2104 vulkan: allocate the device fault vendor binary buffer (#32008)
03d6c82 tests : use only the public ggml API (#32077)
adde94c ui: open the virtual keyboard in the chat editor on Android (#32060)
459ab1c cuda : use MMVF for BF16 matrices of few rows at small batch sizes (#31959)
77e2db8 cuda : add sm70 FlashAttention config case for DKQ=256, DV=256, ncols=64 (#27997)
9d7b2d7 cuda : fattn mma head sizes 40 72 (#30285)
9e9619d CUDA: Add OCP FP8 support (#28413)
f3b407c sycl : fixed sycl CLAMP to actually support strided view (#31784)
aa75ff3 chat : fix Muse Spark ignoring response_format json_schema (#32017)
b5cb8f6 ui: keep a hidden sidebar from intercepting clicks again (#32035)
85822a7 ui: update the README banner to the new llama.cpp cover and remove the legacy brand assets (#31789)
2b66bfb ui: show audio in Discover for models whose mmproj has an audio encoder (#31837)
582eab7 ci : bump ty to 0.0.100 (#32030)
7c2653a mtmd: fix string format, rename to mtmd_format (#32040)
7592b61 ui: move Models, MCP and Settings to a sidebar footer (#32022)
cd4ed90 cuda : remove unused ids and type_src0 parameters (#32018)
202c1a2 llama : Add FP8 KV-cache support (#28392)
e24d0b4 Merge pull request #32015 from ggml-org/gg/fix-sampling-output-aliasing
3e86ce2 llama : reuse backend scheduler when sched_params are compatible (#32009)
494d0f4 ggml : add MUL_MAT src1 precision tests and gating (#31896)
281e7da ui: gather chats into named groups in the sidebar (#32012)
90e3841 ui: show settings save button even while focused (#32010)
c75fba3 cuda : remove unused quantize_cuda_t typedef (#31884)
40f4f4a graph: mark every sampling chain as an output to prevent aliasing
54cde71 ui: expand the chat form into a large editor for longer messages (#31973)
7d6fd84 convert : mark Sarvam byte fallback tokens as BYTE (#32006)
4ed88da convert : workaround conversion for p-e-w/gpt-oss-20b-heretic-ara-v4 (#31860)
ac931b5 cuda : halve VMM pool reservation size and retry on cuMemAddressReserve failure (#30542)
15759e0 CUDA: skip graph compat check when graph is being reused (#32004)
a591cbe mtmd: share compute buffer among subcontext (#31945)
203b5f4 server: fix output lang for realtime api (#31982)
351025e mtmd: add kimi K3.1 vision (#31980)
0d9eaca ggml : fix FreeBSD Vulkan build by setting _XOPEN_SOURCE=700 (#31701)
17701e4 CUDA: move graph and concurrent stream state out of common.cuh (#32001)
ac00705 ui: use a live-preview markdown editor for the chat form input (#31955)
940a30e model : stop renaming the recurrent cache tensors in plamo2 (#31984)
45c8ea8 common/json-schema : fix allOf with plain types (e.g. integer) generating an empty object grammar (#31834)
c90aa99 common : escape control characters in tool argument strings (#31937)
a2c942d cann: fail early when checking if buffer type is supported (#31270)
18c5a95 sycl: Add device-side MoE scheduler (#30156)
ae4893a vulkan: add tiled-transpose concat shader for Qwen models (#30591)
6122b56 ggml-rpc : fail gracefully on unreachable endpoints and failed handshakes (#30415)
3837bf0 vulkan: Divide GROUP_NORM workgroup to 2D to avoid maxComputeWorkGroupCount limit (#31930)
c1a4feb vulkan: Fix out of bounds reads/writes on MoltenVK (#31890)
78ffbaa vulkan: cap scalar flash attention at 8 rows for large heads on Qualcomm (#30829)
58e38bd llama : apply the iGPU lazy-mode fallback only when mmap is off (#31957)
d5bbcd1 ci: update oneAPI 2026.2, oneDNN 2026.1 and Level Zero SDK 1.34.0 (#31540)
8766c02 sycl: un-gate oneDNN fused SDPA for Alchemist at head_dim=64 (#31574)
3b4d6ab chat : don't use lazy grammar when response_format is set with tools (#31956)
cf6463a hexagon: fix 64-bit FastRPC handle truncation on close (#30952)
711f9f4 server: responses api: support conversion of text.format.json_schema to completions (#26145)
26d52b0 server : use common_chat_input in realtime api (#31978)
08f945f server: add oai-compat /v1/audio/speech endpoint (#31763)
20c2e96 mtmd: support gemma 4 input audio stream (#31965)
3d94ffe mtmd: add mtmd_helper_resampler (#31964)
0a94a3f ui : retry opening the sidebar when a demo imports its traces (#31952)
610f7ab ui: make the app feel native on phones (#31949)
5f490d7 feat: external PLE file loading (#31185)
9d340ec Windows unbuffered model load (#26014)
359dc58 server : functionality test for new spec dec (#31027)
a4d4f2f ggml-zdnn: accept non-zdnn tensors, invalidate compute tensors after every op (#31824)
1945793 server : fix response_format json_schema when schema is not nested in the json_schema wrapper (#28697)
0eebb6e ggml : implement the backward pass for TRI (#30712)
d628a98 tests : keep dummy ffn.act scales away from zero (#31940)
a8fad21 mtp: read the target hidden states from a device buffer (#31927)
b39680c ui: fold, preview and rearrange the sidebar's sections (#31936)
806b3c2 ggml-backend: don't upload inputs that are not read (#31419)
0c6dca2 models : Attach Granite NVFP4 weight scales (#31449)
fc0be86 ci : Run gfx1151 ROCm job on Strix Halo Devlab runners (#31567)
e01c9f1 metal : prevent out-of-bounds writes to the Metal FA output buffer (#31662)
4b46d96 metal : make ggml_tri select instead of scaling by the comparison (#30711)
f317836 opencl : include climits (#31935)
2d99adb mtmd: Pocket TTS - add generic support for temperature set via API (#31655)
cf92c13 RPC: Improve loading times (#29413)
9f0d80c ggml : support YaRN truncation config on openvino, sycl, vulkan, webgpu, rpc (#30802)
a0ad695 sycl: cache device-id (#31126)
6590d25 tests: bound the test_rope tolerance by the f32 angle resolution (#31874)
53eb669 opencl: add f32 MUL_MAT_ID (#31783)
d78f425 ggml-zdnn: fix release binary not detecting backend lib (#31905)
7d7f028 mtmd: refactoring: remove clip.h, consolidate code into mtmd (#31865)
b65bf69 cpu: enable SpacemiT IME2 MXFP4 dense repack (#31776)
a3c7cf0 llamafile: impl bf16 and q8_0 vxe (#31866)
57d0482 ggml : update CUDA/Metal radix Top-K to make tied selections deterministic (#31084)
9982004 ggml-cuda : add d_state=64 fast path to SSM_SCAN kernel (#31818)
cf34d31 moe-cache : ensure CPU part does not wait for the cached experts (#31888)
a691b23 webgpu : add COUNT_EQUAL support (#31578)
2277e75 ci: move runners to Ubuntu 26.04 (#31529)
12ab597 ggml : switch to F32 src1 in the CPU backend (#31854)
d6be065 llama, mtmd: follow the reference precision of ggml_prec_set_acc calls (#31851)
a2b3018 speculative : do not apply the MoE cache to the draft model (#31497)
c19043f opencl: improve q8_0 and MoE small-n verification for spec (#31831)
372dc5c common : sanitize invalid UTF-8 in PEG tool call arguments (#31861)
f477f79 server : downgrade per-request proxy logs to trace (#31859)
76a6773 hexagon: support for automatic loading of libhtp without legacy env vars (#31781)
12e9084 sycl: exact rounding in CPY to Q4_1, Q5_1 and Q8_0 (#31465)
818ac6b sycl: refuse device buffers that would exceed VRAM (#31285)
e9a2a44 hexagon: n_hvx and n_threads cleanup (#31857)
a2654ec mtmd: move log-mel computation to cgraph (#31808)
9f0e509 mtmd: support MageVL-Cowork model (#31819)
e95426f llama_moe_cache: run missed experts on CPU, add tuning script (#30966)
f7591a6 sycl: add f16 support for SWIGLU_OAI (#30857)
80d2c0c metal : fix GROUP_NORM for ne03 > 1 and non-contiguous inputs (#31852)
0956684 server: visit parent slots before child slots (#30779)
cf659f2 webgpu : Implement group norm kernel (#31490)
1938152 vulkan: Add 'readonly' to decodeBuf declarations. (#31833)
5d7d4ae CUDA: fix shared memory bank conflicts in the lightning indexer (#31803)
c0ab6e5 llama : add missing link to pthread (#31785)
31424b7 opencl: add arange (#31829)
fd27b20 meta : raise the compute view limit for buffers with few tensors (#31447)
ca89968 hexagon: fuse post-norm residual add into RMS_NORM_MUL (#31822)
bcb0152 model: add Kimi-K3.1-Pro text model (#31779)
669204e hexagon: conv_2d optimizations and support for conv_3d (#31814)
954de24 docker: add explicit /usr/local/nvidia search path (#31806)
6c01da6 cli : fix crash on invalid UTF-8 in prompt or files (#31563)
8820b08 tests : qualify isnan and signbit with std:: (#31791)
9a1d03c tests : override hardware_concurrency in CI (#31797)
18ab62a cmake : generate RC file directly without win-app.rc.in (#31718)
52c9bad opencl: support repeat for f16, i32, i16, bf16 (#31782)
5d3ed2a webgpu: add batched tensor API (get_tensor_n and set_tensor_n) (#31473)
80088fa vulkan: detect ReBAR support, avoid BAR allocations if unavailable (#31658)
c4fb579 moe-cache : LRFU/MQ-like hybrid cache, ClassLRU (#31233)
3ed7284 ci : bump CUDA 13 release builds to 13.5 (#31788)
d9cf187 vendor: update BoringSSL to 0.20261105.0 (#31757)
240d68f CUDA: fix Maxwell build (#31787)
bbc2e4b common : restrict HF token to huggingface.co hosts (#31730)
4b77dee opencl: add q6_k multi column tiled GEMV for speculative verification and refactoring (#31758)
67107ac hexagon: Implement set_tensor_n and improve repacking/tiling robustness (#31744)
e14a779 mtmd: enable cmake unity build (#31760)
d38bea3 tests : drop u8path() (#31755)
186eab1 ci: fix build-cmake-pkg ccache (#31705)
6023692 llama: fix kv shifting, add tests, add per-model build_rope (#31683)
b1c902f llama, mtmd: (chatterbox) add use_classifier_free_guidance as gguf metadata (#31698)
e8df20d models : recursive fix (#31749)
fecdc81 model : add Ripple (ricursiveai) (#31692)
45b6604 model: Add support for the xing4_0 (#31589)
e30ad80 ui: prevent a hidden sidebar from intercepting clicks (#31712)
e0f96a1 llama: refactoring: move model to class member of llm_graph_context (#31733)
468fde9 server : simplify windows MCP command resolution (#31728)
1416f76 vendor : update cpp-httplib to 0.65.0 (#31731)
10d4846 ggml-cuda: sanitize post-bias NaNs in MoE selection (#31005)
61d4472 vulkan: Implement test-fusion interface (#30769)
016c1d9 CUDA: use highest compiled arch in get_mmvq_mmid_max_batch (#31679)
da3416d vulkan: RDNA3/4 Large-Tile Split-K Optimization (#30280)
cc78da9 gguf-py : external chat templates have precedence over tokenizer_config (#31717)
ea40a3a metal : fuse DeepSeek-V4/V4.1 and GLM MoE routing (#31615)
15e5099 CUDA/HIP: matrix-core (MFMA) lightning indexer for CDNA2 (gfx90a) (#29050)
69ee2ec CUDA: linearize MoE data layout for MMQ/MMF (#31047)
2bc46ba vulkan: FWHT kernels for block widths above 512 (#29772)
3b83d12 vulkan : check combined K+V f16 scratch size in FA dequant path (#30487)
e152332 llama-server: Use reference-counted checkpoints instead of deep-copies (#30137)
1fa7f1c ggml : support truncate=false for YaRN (#30206)
9c9074e conversion : resolve quantized_layers from source tensor names (#31611)
f7b41cc metal : use mul_mm for 32 <= K < 64 (#30982)
c55b094 server : reject non-object session updates (#31714)
2c19244 fit : account for skipped NextN layers when setting n_gpu_layers (#31569)
553d2c2 webgpu : add MEAN support (#31591)
040ce19 graph: store the MTP conv snapshots of every recurrent model in one set_rows (#31660)
d4c345d ci : remove unused model generation (#31412)
2ada830 common : drop u8path() and use generic_string() (#31681)
5a081c0 server : drop u8path() and use generic_string() (#31682)
7e42b95 rpc : align code with common.cpp (#31684)
3259ab6 vulkan: Re-enable large coopmat matmul configuration on AMD proprietary driver 26.10.1+ (#31587)
a01dff9 tests: fix CI flakes of TOPK_MOE caused by near ties (#31426)
44f91a0 metal: keep fusable node groups inside one command buffer (#31688)
330f018 qwen2 : add nvfp4 scales to build_ffn (#31657)
7d516e2 opencl: enable f32 and q5_k multi column GEMV by default for Adreno X2 (#31694)
8bed7cf opencl: support non-contig sigmoid (#31687)
8415bd7 opencl: add DSV4_HC_PRE and DSV4_HC_POST (#31623)
5ce06fc Title: CUDA: fuse NVFP4 gate/up matmuls and GLU in MMQ (#31296)
e56f006 llama : fix on-device state copy sizes for quantized KV caches (#30541)
3de52ac ci: run self hosted cpu jobs on hf or github (#31439)
6f1e654 server : realtime: send an error event instead of crashing on exceptions (#31632)
9e2a04a opencl: improve decode perf with FA (#31633)
c6a0ff1 ui: open actions on right-click for chats, tabs, messages and models (#31545)
5851a8e model, mtmd: drama-3 tts support (#31613)
efbf6e8 ui: show text and page thumbnails for PDF attachments, stop endless conversion retries (#31468)
b393d73 webgpu: return maxBufferSize from get_max_size and avoid whole-buffer bindings in memset/clear (#31348)
00df7c2 sycl : fix docker issue (#31605)
034e253 gguf-py : validate every tokenizer merge pair, not only the first (#31631)
52e53a7 common : delete common_get_path_from_env() (#31444)