ggml-org/llama.cpp b11103

on GitHub

spec : support DFlash for HunyuanOCR (#28890)

• model : add DFlash layer-input taps for HunyuanVL

DFlash speculative decoding needs the target graph to expose the residual
stream entering each layer (res->t_layer_inp[il]) - the draft model reads
those tensors to build its cross-context. Qwen3 and the other DFlash-capable
targets register them, but the Hunyuan graphs do not, so serving a DFlash
draft against a HunyuanOCR target aborts during the first graph build:

GGML_ASSERT(t_layer_inp[il] != nullptr && "layer input tensor is null")

Register the tensor at the top of the layer loop, mirroring qwen3. The
layer input is the residual stream entering layer il, i.e. the output of
layer il-1, which is what the draft's target_layers metadata refers to
(the converter writes target_layer_ids+1). hunyuan-dense.cpp reuses this
graph, so it is covered as well; hunyuan-moe has a separate graph and is
untouched.

The vector is only read when a speculative implementation enables those
layer ids, so there is no behaviour change without a draft model.

Tested with tencent/HunyuanOCR 1.5 and its DFlash draft: image requests now
run, draft acceptance is ~0.5 and the OCR output is byte-identical to the
non-speculative run.

Co-authored-by: wendadawen wendadawen@qq.com

• convert : fix DFlash draft conversion against HunYuan targets

Converting a DFlash draft with a HunYuan target failed in two ways.

1. DFlashModel.set_vocab() reuses the target class' vocab handling by
   calling it unbound with the draft instance, but HunYuanModel.set_vocab()
   called self._fix_special_tokens(), a method that only exists on
   HunYuanModel, so the conversion always aborted with

   AttributeError: 'DFlashModel' object has no attribute '_fix_special_tokens'

   Make the vocab helpers module-level functions taking the model
   explicitly, so they do not depend on the instance being a HunYuanModel.
   They have no other callers, so the two id lookups are folded into
   _fix_special_tokens().

2. The delegated call runs with self.dir_model pointed at the target but
   keeps the draft's self.hparams, so config lookups inside the target's
   vocab code (the pad_token_id < 0 guard, eod_token_id) read the draft's
   config instead of the target's. That aborts on targets with
   pad_token_id = -1 (e.g. the HunyuanOCR v1.0 checkpoint) and otherwise
   writes special token ids that disagree with the target.

   Add _vocab_hparams(): it returns the target's config (with text_config
   merged to the root, as TextModel does) when the model is a draft
   converted with --target-model-dir, and the model's own hparams
   otherwise, so a normal conversion is unaffected.

Tested: converting tencent/HunyuanOCR/dflash succeeds with both the 1.5 and
the v1.0 target; converting the base model without --target-model-dir
produces a byte-identical GGUF to before.

Co-authored-by: wendadawen wendadawen@qq.com

• convert : fix DFlash draft vocab against HunYuan targets

Switch hparams to the target config for the duration of the borrowed
set_vocab(), matching the existing dir_model swap, instead of teaching
HunYuanModel::set_vocab about draft models.

• convert : fix HunYuan special token ids for DFlash drafts

• convert : use load_hparams for HunYuan special token ids