ggml-org/llama.cpp b8560

on GitHub

server : add custom socket options to disable SO_REUSEPORT (#21056)

• server : add custom socket options to disable SO_REUSEPORT

Signed-off-by: Adrien Gallouët angt@huggingface.co

• Add --reuse-port

  $ strace -e trace=setsockopt,bind build/bin/llama-server -lv 2 --reuse-port
  setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
  setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  setsockopt(3, SOL_SOCKET, SO_REUSEPORT, [1], 4) = 0
  bind(3, {sa_family=AF_INET, sin_port=htons(8080), sin_addr=inet_addr("127.0.0.1")}, 16) = 0

  $ strace -e trace=setsockopt,bind build/bin/llama-server -lv 2
  setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
  setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
  bind(3, {sa_family=AF_INET, sin_port=htons(8080), sin_addr=inet_addr("127.0.0.1")}, 16) = 0

Signed-off-by: Adrien Gallouët angt@huggingface.co

• Update tools/server/README.md (llama-gen-docs)

Signed-off-by: Adrien Gallouët angt@huggingface.co

• Fix windows

Signed-off-by: Adrien Gallouët angt@huggingface.co

Signed-off-by: Adrien Gallouët angt@huggingface.co