github ggml-org/llama.cpp b12122

pre-releaseone hour ago
Details

Title: CUDA: fuse NVFP4 gate/up matmuls and GLU in MMQ (#31296)

  • cuda: fuse NVFP4 gate/up and GLU in MMQ

  • cuda: honor the requested src1 precision in fused gate/up MMQ

  • cuda: share NVFP4 scale checks, skip unused fused FP4 kernels

  • cuda: test NVFP4 fused MMQ at W4A8 without scales and with batches, clean up naming

  • cuda: check gate_scale before loading the fused NVFP4 gate scale

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.