ggml-org/llama.cpp b11985

on GitHub

metal : faster MXFP4 mat-vec, full threadgroups for unary/binary/GLU (#31075)

• metal : faster MXFP4 mat-vec kernel

Decode the fp4 values with the fp16 bit trick instead of a threadgroup lookup table, and load the packed nibbles as aligned 32-bit words. The kernel no longer uses threadgroup memory.

• metal : launch contiguous unary and binary ops with full threadgroups

The contiguous unary path and the row-broadcast binary path used one thread per threadgroup. Use up to 256 threads per threadgroup and bound-check the tail.

• metal : split GLU rows across threadgroups

A GLU row ran on one threadgroup, so single-row decode used a single threadgroup for the whole op. Split each row over threadgroups of up to 256 threads.