github ggml-org/llama.cpp b11190

latest release: b11191
pre-release2 hours ago
Details

mtmd: fix mel preprocessor in LFM2 audio (#29403)

which resulted in different greedy transcripts for 4.5% of English and 6.5% of Japanese
test utterances. In Japanese, some differences changed entire words.

This change:

  • uses log(x + 2^-24) instead of clamping to the log floor
  • uses a symmetric Hann window, equivalent to torch.hann_window(periodic=False)
  • adds the normalization epsilon to the standard deviation instead of inside the square root

Only the lfm2a preprocessor opts into these behaviors. Other audio preprocessors are unchanged.

Tested on top of 84e76d8 using llama-server with CUDA and temperature=0, compared against
http://github.com/Liquid4All/liquid-audio fp32.

Test set:

  • 200 LibriSpeech test-clean utterances (EN)
  • 200 Common Voice ja test utterances (JP)
  • identical 16 kHz audio passed to both implementations
Greedy transcript identical to liquid-audio Without fix With fix
EN F16 191/200 200/200
JP F32 187/200 200/200
JP F16 187/200 199/200

The remaining JP F16 difference is a comma and matches the reference implementation's own bf16
output.

Mel relative L2 error versus liquid-audio:

  • EN: 3.2% -> ~2e-6 median
  • JP: 3.9% -> ~2e-6 median

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.