ggml-org/llama.cpp b8102

on GitHub

model : add tokenizer from LFM2.5-Audio-1.5B (#19687)

• model : Add tokenizer from LFM2.5-Audio-1.5B

LFM2.5-Audio-1.5B introduced lightweight audio tokenizer.

Tokenizer based on LFM2 architecture and acts as "embedding" model with
different input n_embd and output n_embd_out.

To be used in #18641.

To convert use

python3 convert_hf_to_gguf.py /path/to/LFM2.5-Audio-1.5B/audio_detokenizer

• Update convert_hf_to_gguf.py

Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@scala.com

• Formatting

• Rework check for attention layers

• Add LFM2 SWA model support

• Address PR feedback

• Set vocab to none

• Move helper function definitions to cpp file

Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@scala.com