ggml-org/llama.cpp b11331

on GitHub

CUDA: Handle compute type for NVFP4 on cublass path (#29173)

• CUDA: Handle compute type for NVFP4 on cublass path

Signed-off-by: ynankani ynankani@nvidia.com

• Use BF16 compute type for quantized models if HW allows

Signed-off-by: ynankani ynankani@nvidia.com

• Set acc prec to bf16 for nvfp4 as it needs atleast bf16 range

Signed-off-by: ynankani ynankani@nvidia.com

• Update ggml/src/ggml-cuda/ggml-cuda.cu

Co-authored-by: Johannes Gäßler johannesg@5d6.de

• preserve op_params for per-expert matmul

Signed-off-by: ynankani ynankani@nvidia.com

Signed-off-by: ynankani ynankani@nvidia.com
Co-authored-by: Johannes Gäßler johannesg@5d6.de