github ggml-org/llama.cpp b9970

latest release: b9971
one hour ago
Details

ggml : add GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer (#24231)

  • ggml : add GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer

  • ggml : remove scale parameters from lightning indexer OP, add f16 mask parameter

  • tests : add GGML_OP_LIGHTNING_INDEXER tests

  • ggml : bump RPC version

  • chore : check if lightning indexer input tensors are not transposed

  • tests : count flops instead of bandwidth in lightning indexer test

  • chore : add missing const

  • chore : whitespace

  • ggml : renamed variables in CPU lightning indexer implementation

  • ggml : fix lightning indexer mask broadcasting

  • tests : tests for lightning indexer mask broadcasting

  • chore : whitespace

  • llama : use GGML_OP_LIGHTNING_INDEXER in DeepSeek V3.2 and DeepSeek V4 models


Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.