ggml-org/llama.cpp b10241

on GitHub

CUDA: Fix data-races when reusing SMEM in block_reduce (#26385)

• CUDA: Fix data-races when reusing block_reduce

block_reduce currently doesn't resync after reading from SMEM, causing
potential data-races when reusing SMEM for multiple reductions.

One may consider simply always adding this in block_reduce, but this
comes at a potential perf cost

• double-buffering for single-row softmax

• double-buffering for norm as well

• Add comment

• Add explanatory comment to block_reduce

• Specify need for + do memory barrier only in multi-warp scenario

• Implement review-suggestion from @gaugarg-nv