github ggml-org/llama.cpp b9084

latest release: b9085
5 hours ago
Details

hexagon: add HTP kernel for GGML_OP_GATED_DELTA_NET (#22837)

Implement the Gated Delta Net recurrence on HVX with:

  • 4-row fused kernels for PP (prompt processing) path
  • 8-row fused kernels for TG (token generation) path, reducing
    K/Q/gate vector reload overhead by 2x
  • Separate PP/TG thread functions for I-cache isolation
  • VTCM state scratchpad with DMA in/out for TG single-cycle access
  • Vectorized gate exp via hvx_exp_f32

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.