github ggml-org/llama.cpp b10772

pre-releaseone hour ago
Details

ggml-hexagon: add F16 support for unary ops (#28228)

Extend the HTP backend's F16 unary op coverage to include ABS on top
of the existing NORM/RMS_NORM/L2_NORM/SCALE/CLAMP/SQR/SQRT set.

  • Add hvx_abs_f16_{aa,au,ua,uu} + dispatcher in hvx-arith.h, mirroring
    the sqr_f16 kernel structure and using the existing hvx_vec_abs_f16()
    sign-bit-clear helper
  • Add abs_f16() row-wise dispatch and DEFINE_UNARY_TASK_F16(unary_abs, ...)
    in unary-ops.c, wired into execute_op_unary()'s op_type/task_func
    switches
  • Register HTP_OP_UNARY_ABS in htp_op_is_unary() (unary-ops.h) so that
    ggml_hexagon_precompute_unary_params() fills kernel_params (n_threads,
    VTCM layout) for ABS nodes -- required for the F16 path to function
  • Narrow the F16 GGML_OP_UNARY gate in ggml_hexagon_supported_unary()
    (ggml-hexagon.cpp) to allow GGML_UNARY_OP_ABS specifically, instead of
    rejecting all GGML_OP_UNARY ops for F16
  • Merge the separate execute_op_unary_f32()/execute_op_unary_f16()
    functions into a single execute_op_unary(), branching on an is_f16
    flag for the parts that actually differ by type (elem_size, the
    early F16 op-support check, and which task_func table to use) while
    keeping the F32-only tiled/RMS_NORM_MUL paths intact -- per review
    feedback to avoid duplicating the shared VTCM/DMA plumbing

Verified on-device (QRD8850, Hexagon v81) via test-backend-ops -o ABS:
8/8 passing (F16 + F32, HTP0, no CPU fallback). Regression-checked
SQR/CLAMP/SQRT (F16+F32) and NORM/RMS_NORM/L2_NORM/SCALE (F32; their F16
paths have no CPU reference kernel in test-backend-ops and cannot be
correctness-tested there independent of this change).

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.