ggml-org/llama.cpp b11163

on GitHub

llama: add llama_batch_ext (#24669)

• (wip) add llama_batch_ext

• wip

• updated design

• updated impl

• change signature

• unused var

• demo common_prompt_batch_decode

• fix pos

• tmp disable test-batch-alloc

• fix compat

• nits: add const

• no more pos_max

• add comment about llama_batch_ext_set_embd_state

• handle n_embd_out properly

• rename api --> embd_token

• llama_embd

• stub llama_batch_ext_set_embd_state

• support both token + embd + state in batch

• llama_batch_ext_add_embd

• upstream some changes

• nits

• fix test-batch-alloc

• add test for compat