github ggml-org/llama.cpp b11822

pre-releaseone hour ago
Details

model : fix qwen4exp layer inputs for DFlash drafters (#30927)

qwen4exp published the hc-wide residual [n_embd, hc, n_tokens] as
t_layer_inp, but the drafter reads layer inputs as [n_embd, n_tokens]
rows: llama_context sizes each embd_layer_inp buffer as n_embd * n_batch
and the DFlash encoder input is n_extract * n_embd wide. Evaluating a
DFlash draft over a qwen4exp target therefore aborts on the first request
in llama_context::extract_layer_inputs.

Collapse the streams with a plain mean before exposing them, as deepseek4
already does for its hyper-connection residual, and build the mean only
for the layers a drafter actually requested.

Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.