ggml-org/llama.cpp b12291

on GitHub

vulkan: allow dequantized KV optimization with CM2 (#31526)

• additional FA test cases

Assisted-by: Codex

• vulkan: allow dequantized KV optimization with CM2

Enable the KV dequantization optimization with coopmat2, guarded by a heuristic
based on query tokens*GQA ratio.

While doing this we saw very large speedups for certain head sizes, and it was
because those head sizes were incorrectly treated as unaligned. So also fix
those to be treated as aligned, which should help for when the dequant
optimization doesn't apply.

• gqa uses nbq2 for the stride, so we need to check that value for alignment

• tests : use VARS_TO_STR18 for flash attention

Assisted-by: Codex