ggml-org/llama.cpp b8469

on GitHub

[CUDA] Increase number of output elements per-thread block if the K-dimension is small (#20635)

• Increase per-thread work if the K-dimension is small

With tensor parallelism, the K-dimension of the FFN-down matrices is split, which makes it quite small, especially for MOEs. For example, Qwen3-30b-A3B has a K-dimension of 768, and Qwen3235B-A22B has k-dimension of 1536.
The current heuristic uses a group of 4 warps irrespective of K-dimension size, resulting in some of the threads being idle. This results in poor performance for these matrices.

This change increases the number of output elements per block for such cases.

• Limit this change to ncols_dst = 1

• tab to space