github ggml-org/llama.cpp b12458

pre-release42 minutes ago
Details

mtmd: support YuE2-3B (#32369)

  • mtmd: support YuE2-3B

  • improve speed

  • add CFG

  • fix streaming

  • add cfg control from api

  • check ctx size

  • fix n_predict

  • add flow_steps config

  • nits

  • fix batch api

  • nits

  • adapt

  • mtmd: yue2 samples from its own saved logits

The context logits belong to the last decode of any slot, so with
several server slots the score and the unguided codes were sampled
from another sequence. Keep the logits of each own decode in
save_logits(), which also does the guidance mix.

  • mtmd: let the yue2 snake chain fuse

The beta reshape landed between sqr and mul, so the mul, sin, sqr,
mul, add chain was not consecutive and the backends never fused it.
Expand both reshapes first, like qwen3tts does.


Co-authored-by: Pascal admin@serveurperso.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.