ggml-org/llama.cpp b11716

on GitHub

hexagon: add support for Q1_0 and Q2_0 (#30122)

• hexagon: add support for q1_0 quant

• hexagon: enable Q1_0 weights in the 2D HMX MUL_MAT path

• hex-q1: cleanup gates, generalize tiling and remove in-loop divs

• hex-tiling: document tiled formats in developer.md and update htp-tensor.h

• hex-q2: add support for Q2_0

• hex-q2: remove stale asserts and speacial cases

• hex-mm: fix MUL_MAT_NX assumption about square tiling, Q1/Q2 have non-square tiles

• hex-q1/2: minor optimizations (unrolls/un-unrolls/...)

• hex-q1: optimize Q1_0 layout to reduce unpack/vrmpy overhead

• hex-q1/q2_0: further optimize vec_dots

Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.com