github ggml-org/llama.cpp b12473

pre-release14 minutes ago
Details

cuda : raise shared mem limit only for the launched kernel (#32392)

This commit removes a call of the CUDA_SET_SHARED_MEMORY_LIMIT macro as
the fallback decision was moved out of launch_mul_mat_q.
launch_mul_mat_q now has the fallback template parameter which was added
in Commit 6eddde0 ("CUDA: refactor MMQ
kernel configuration (#24127)").

This code was then refactored in Commit
c49edf0 ("CUDA: fuse FFN gate/up
matmuls and GLU in MMQ with interleaved gate/up rows (#29948)") which
added the has_fusion template parameter but it kept both macro calls.

launch_mul_mat_q only launches one type of kernel now (this is now
decided earlier in the templated call chain) and each specialization
will set the limit itself.

The motivation for this is to simplify the code and improve readabilty.

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.