ggml-org/llama.cpp b11095

on GitHub

hexagon: new HMX-optimized GATED_DELTA_NET (#29199)

• hex-gdn: start putting together HMX support for GDN

• hex-gdn: working hmx but not-pipelined and slow for now

• hex-gdn: re-write vtcm layout handling and prep for pipelining

• hex-gdn: starting to pipeline hmx and dmas

• hex-gdn: add hvx threading for most pipeline stages

• hex-gdb: add detailed trace events

• hex-gdn: vectorize expfs and use aligned hvx reads/writes

• hex-gnd: vectorize the rest of expf

• hex-gdn: optimize tail processing (pad partial chunks)

• hex-gdb: avoid float up/down casts in hot loops

• hex-fa: remove float up/down casts from inner loops

• hex-gdn: do exp() in f16 to improve HVX utilization

• hex-gdn: optimize tiler

• hex-hmx: bump hmx-queue to 128 and dispatch all GDN gemms at once

• hex-gdn: further pipeline improvements

• hex-gdn: optimize gdn prep stage

• hex-gdn: yet more tweaks to optimize GND_SOLVE task and pipeline

• hex-gdn: improve accuracy and optmize gdn-prep further

• hex-gdn: fix rebase conflict

• hex-bufs: revert max_bufsize enforcement, it is enough to just enforce max_vmem

• hex-scripts: improved inspect script to avoid false alarms in reg spill detector

• hex-fa: improve inline softmax with in-reg VKQ32 accum

• hex-fa: minor improvement for dma pipeline in hvx kernel

• hex-fa: reduce ddr reads by 20-30% during token gen

• hex-gdn: proper alignment for hvx vtcm spads