github ggml-org/llama.cpp b11744

latest releases: b11747, b11748
pre-release2 hours ago
Details

server: add --connect, allow p2p access to server outside LAN (#28277)

  • init

  • wip

  • wire it up

  • make it clear that it's experimental

  • add props.connect_code

  • nits

  • fix windows build

  • server: do not advertise a connect code without the tunnel

--connect-code on its own had no effect other than being served in
/props, so the Web UI showed a remote access code that no tunnel was
serving. Clear it when --connect is off, and say so once at startup.

The teardown comment claimed that terminate() closes the child stdout.
It does not: the log thread leaves fgets() only once every writer of
that pipe is gone, so a descendant of llama-connect would block the
join forever. Describe the real contract instead.

Also restore the --reasoning-preserve default in the generated docs.

  • Update tools/connect/CMakeLists.txt

Co-authored-by: Adrien Gallouët angt@huggingface.co

  • server: dial llama-connect on the bound port and a reachable host

Pass ctx_http.port so --port 0 and the GCP override work, pick the
dial host from --host (wildcards map to loopback, IPv6 is bracketed)
and refuse --connect when only unix sockets are bound.


Co-authored-by: Pascal admin@serveurperso.com
Co-authored-by: Adrien Gallouët angt@huggingface.co

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Don't miss a new llama.cpp release

NewReleases is sending notifications on new releases.