ggml-org/llama.cpp b11160

on GitHub

vulkan: int8 coopmat1 matmul implementation for AMD RDNA3 and RDNA4 (#27952)

• vulkan: add int8 coopmat quantized matmul shader

• apply scales inline

• use scalar sums

• probe and directly access coopmat values instead of going through shmem

• add q8_0 support

• add BK_STEP to shader, default to 2

• use larger workgroups

• double buffering

• preload scales

• coopmat load first, then wmma

• use float for scales

• add faster RDNA int->float conversion

• workgroup scheduling for cache proximity

• clean up

• use wave32

• restructure for vgpr use

• skip computation for inactive tiles

• only force subgroup size 32 on AMD RDNA

• use BK_STEP 4

• fix compilation

• move quant-specific prefetch function out of main file

• add q4_1, q5_0, q5_1 support

• restructure mmq cm1 functions

• enable mul_mat_id support

• fix segfault

• fix mul_mat_id bug

• support iq4_nl and mxfp4

• remove elem row/col fast path, invalid for RDNA4

• use shmem arrays for LUTs

• use 4-byte loads where possible

• add q3_k, q4_k, q5_k, q6_k and nvfp4 support

• fix l warptile

• improve performance

• improve performance

• improvements

• dedup b scales

• merge shmem arrays

• undo uint8_t, gate to RDNA3/4

• add RDNA4 architecture, use for hardcoded coopmat elem thread access, set BK_STEP back to 4

• improve offset application

• clean up

• fix iq4_nl and nvfp4 performance

• rdna4 tuning

• use BK_STEP 2 on MUL_MAT_ID

• adapt to upstream changes

• fix shmem support function, clean up comments

• fix warptile logic

Co-authored-by: Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com

• vulkan: add IQ4_XS support to the coopmat1 integer matmul shader (#28440)

Adds IQ4_XS to mul_mmq_cm1: dedicated block_a_load/block_a_to_shmem that
expand both nibbles of each packed32 word through cm1_kvalues, LOAD_VEC_A 8
and an IQ4_XS-sized a_panel_bytes estimate for the L2-friendly scheduling.

Assisted-by: OpenAI Codex

Co-authored-by: Claude Opus 5 (1M context) noreply@anthropic.com

• avoid compiling f16 acc shader variants

Co-authored-by: Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com
Co-authored-by: Claude Opus 5 (1M context) noreply@anthropic.com