github ggml-org/llama.cpp b12554

latest releases: b12555, b12556
pre-release57 minutes ago
Details

ggml : add BF16 support to ROPE and SET_ROWS (CPU, CUDA) (#32387)

  • ggml : add BF16 support to ROPE and SET_ROWS (CPU, CUDA)

  • cuda : SET_ROWS from BF16 into quantized types and F8_E4M3

  • cuda : fuse BF16 rope into a BF16 cache

  • tests : trim the BF16 rope cases to one per code path

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.