github ggml-org/llama.cpp b10016

latest release: b10017
3 hours ago
Details

[SYCL] Flash Attention with XMX engine via oneDNN (#25222)

  • [SYCL] F16 (default) Flash Attention with XMX engine via oneDNN graph API; Qwen3.6-27b-Q8_0 prefill speed up x1.21 at p=512 and x4.26 at p=80k

  • [SYCL] Address review on FA oneDNN path. Result: llama-bench---pp512; 32% increase with fa1; llama-perplexity---0.11% difference; tested model: mradermacher/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf

  • PR-25222 revision v2: addressed audits

  • [SYCL] flash-attn oneDNN SDPA KV F16 rev 3.0: add BMG gate + multi-device sync. Narrow the scrope of this PR to Battlemage only (bmg; Xe2). Other archs (e.g., alchemist) fall back to existing FA kernel. When device_count >1, apply stream -> wait_and_throw(), validated working path for multi-gpu sync fix by @maxious.

Co-authored-by: maxious 81432+maxious@users.noreply.github.com

  • updated comment on bmg gate, noted the issue

Co-authored-by: scientist3 scientist.3@users.noreply.github.com
Co-authored-by: hmscider hmscider@users.noreply.github.com
Co-authored-by: maxious 81432+maxious@users.noreply.github.com

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.