github ggml-org/llama.cpp b12376

pre-release42 minutes ago
Details

Hexagon: add support for IQ2_XXS and NVFP4 (#31954)

  • ggml-hexagon: IQ2_XXS and NVFP4 tiled repack layouts

  • ggml-hexagon: IQ2_XXS HVX matmul kernel

  • ggml-hexagon: NVFP4 HVX matmul kernel

  • ggml-hexagon: NVFP4 per-tensor scale

  • ggml-hexagon: HMX paths for IQ2_XXS and NVFP4

  • ggml-hexagon: supports_op coverage for NVFP4 and IQ2_XXS

  • fix HMX IQ2_XXS dequant scale underflow in fp16

  • hexagon: IQ2_XXS 2-gather tile decode with compact grid, small-weight tests

  • IQ2_XXS vpopcount sign parity and gather pipelining

  • pipeline IQ2_XXS HMX dequant gathers and add large_w tests

  • hex-iq2-xxs: rebase and cleanup

Remove unused helper functions.
Add lut_size to the overall vtcm required size.
Trim down (way way down) the new tests.

  • hex-iq2-xxs: actually trim down the tests

Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.