ggml-org/llama.cpp b12398

on GitHub

vulkan: optimize prompt processing Flash Attention for RDNA3 and RDNA4 (#31323)

• vulkan: optimize Flash Attention cm1 for RDNA3 and RDNA4

• reduce shmem use through buffer reuse

• clean up and fixes

• limit q_in_regs to prefill

• vulkan: add flipped FA path, each subgroup owns 16 Q rows

• cleanup

• move flip shader into own file

• replace kvsep with direct load mode

• fix nvidia hang, tune AMD

• only store pv in sfsh if necessary

• fix skip condition

• nvidia tuning

• reduce occupancy on RDNA4 for large attention head sizes

• fix sparse FA fp16 underflow bug