ggml-org/llama.cpp b8911

on GitHub

hexagon: add support for basic and extended Op profiling (#22269)

• hexagon: restore HTP_OPMASK_QUEUE

• hexagon: honor OPMASK_SKIP_COMPUTE in hmx-matmul

• hex-prof: restore op profiling

• hex-prof: enable PMU

• hexagon: simplify and improve op-queuing with full profiling support

Add separate profile descriptors.

• hexagon: remove opsync and rename opmask into opstage

opsync is no longer needed since the profiler is fully async now.
opmask name was confusing and opstage is more accurate.

• hexagon: refactor opbatch queue handling

• hexagon: add iface hooks for enabling profiler from the host

Also move all the PMU setup stuff out of the hex-utils since it's not inteded for normal use.

• hexagon: make profiler mode configurable

On older devices getting PMU counters is expensive so it's now optional.

• hexagon: add support for setting profiler pmu events from env

• hexagon: simplify profiler output (no need to print buffs, etc)

• hexagon: simplify pmu counter formating

• hexagon: add a simple profile post-proc tool

• hex-prof: add support for reading logs from stdin

• hexagon: document GGML_HEXAGON_PROFILE

• hex-prof: update default width for dims field

• hex-prof: fix linter warnings and errors

• Update ggml/src/ggml-hexagon/htp/htp-ops.h

Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@scala.com

• Update scripts/snapdragon/ggml-hexagon-profile.py

Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@scala.com

Co-authored-by: Trivikram Reddy tamarnat@qti.qualcomm.com
Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@scala.com