ggml-org/llama.cpp b11647

on GitHub

vulkan : copy strided f16 KV once on AMD so reads spread across memory channels (#27703)

• vulkan : copy strided f16 KV once so reads spread across memory channels

Assisted-by: Claude (Opus 4.8 / Opus 5)

• tests : cover the f16 KV copy to scratch

Assisted-by: Claude (Opus 4.8 / Opus 5)

• vulkan : gate the f16 KV scratch copy to AMD

Assisted-by: Claude (Opus 4.8 / Opus 5)

• vulkan : reuse the generic copy shader for the f16 KV scratch fill

The f16 clause needs only a strided copy, not a dequant, so drop the
dedicated shader and dispatch the existing copy pipeline instead. The
copy is described in source memory order so the strided access lands on
the writes, which stream, instead of the reads, which would concentrate
on a few memory channels.

Assisted-by: Claude (Opus 4.8 / Opus 5 / Fable 5)

• vulkan : extend the strided KV scratch copy to bf16

Assisted-by: Claude Opus 5

• vulkan : clean up comments

Assisted-by: Claude Opus 5

• consolidate copy shaders

• disable on RDNA4

Co-authored-by: Ruben Ortlam rortlam@redhat.com