v0.6.10.0 — llama.cpp becomes the default AI engine
The NVIDIA Linux packages now bundle libkeryx-llama.so — an in-process llama.cpp engine (pinned b10015, CUDA) that replaces candle as the default for everything AI:
- Single VRAM copy (zero-dup): llama.cpp hosts the model on the inference GPU; the PoM walk gathers directly over its resident tensors. The 70B very-high tier runs on a 32 GB card with ~4.5 GB headroom (measured: 27.6 GB total on an RTX 5090, model load ~9 s).
- OPoI inference runs through llama.cpp — faster quantized inference than candle, and new GGUF model architectures are supported as soon as llama.cpp supports them (no longer blocked on candle implementing an arch).
- candle is now a dormant fallback, still compiled in: if the bundled engine can't load (or you delete the .so), the miner transparently falls back to the exact previous candle behavior. candle will be removed entirely in a future release.
- Walk/consensus untouched and guarded: gather is byte-gated against the possession index at every startup (129 sampled chunks) plus the existing N-guard; the full-model byte-identity of llama-resident tensors vs the on-disk GGUF was verified with a 4.9 GB memcmp.
- Env knobs:
KERYX_LLAMA_SO(engine path override),KERYX_LLAMA_CTX(context size, default 4096; use 2048 on tight 70B/32GB setups),KERYX_LLAMA_CUDA_DEVICE/KERYX_LLAMA_SERVER+KERYX_LLAMA_PORT(subprocess fallback).
Validated live: 70B-on-32GB + mixed multi-GPU rig, packaged-tarball runs, 140+ shares 0 rejects, per-card walk rates identical to previous releases (the walk is memory-bound — the wins are VRAM and inference speed).
Windows: the in-process engine is Linux-only for now — Windows stays on candle (unchanged); a Windows DLL build is a follow-up. AMD: unchanged (Vulkan llama-server as before); AMD Linux packages follow separately.
NVIDIA Linux assets
12 tarballs (modern sm_75+ / legacy sm_70+ / pascal sm_60-61) + per-line SHA256SUMS. Package sizes grew by the bundled engine (~200 MB–1 GB per line). Windows zips + macOS tarball attached by CI.
Package versions: CUSTOM_VERSION 0.6.10.0 / BUILD 27; banner/user-agent now 0.6.10.