ggml-org/llama.cpp b12418

on GitHub

ggml-cpu: add x86 VNNI Q2_0 dot product (#26348)

• ggml-cpu: use VNNI for Q2_0 dot products

Keep the upstream AVX2 unpacking, accumulation, and fallback. Use VNNI for the integer dot product on supported x86 CPUs.

Assisted-by: Codex

• ggml-cpu: use VBMI for Q2_0 unpacking

Use multishift to unpack Q2_0 weights on CPUs with AVX-512 VBMI and VL. Retain the existing shift-and-mask unpacking on other CPUs.

Assisted-by: Codex

• Update ggml/src/ggml-cpu/arch/x86/quants.c

Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co

Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co