github ggml-org/llama.cpp b11776

latest releases: b11780, b11778, b11777...
pre-release2 hours ago
Details

server: Support logprobs with speculative decoding (#27196)

  • return logprobs in spec path

  • server: add regression test for logprobs with speculative decoding

Asserts accepted draft tokens report the target model's logprobs,
comparing a draft-enabled run against a no-draft baseline (#24271).
Fails on master (top_logprobs empty, logprob 0.0), passes with the
fix from #27196.

Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01KARBWpPcijcBXzJEvttuwk


Co-authored-by: Jeonghyun Lee sig.jhl@gmail.com
Co-authored-by: Claude Fable 5 noreply@anthropic.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.