Anbeeld/beellama.cpp v0.4.8 on GitHub

Changelog

Updated the llama.cpp base from 465e49b9c to c06f84160 and ggml to 0.26.0. Inherited changes include GLM-5.3-Flash, Maple, HrmText, Ling 3.0 VL, and decision-model support; expanded MTP/DFlash, probabilistic drafting, sparse FlashAttention, multimodal embeddings, /v1/systemone, extended batch APIs, imatrix calibration, and backend/server/Web UI fixes. Session/sequence-state formats are now 11/4, the backend API is 3, and RPC is 7.0.0. Legacy mmap/mlock/direct-I/O switches are replaced by --load-mode.
Added PrismML Bonsai 2 folded-weight support. Validated prism.hadamard.* metadata drives activation-side sign/rotation transforms and inverse token-embedding transforms, including Qwen3.5 MTP; graph validation rejects folded weights consumed without their required transform instead of silently producing incorrect output. Added group-128 PQ2_0 and PTQ1_0 weight formats with CPU and CUDA quantization, dequantization, row lookup, matrix-vector/matrix-matrix kernels, GGUF/Python type plumbing, and llama-quantize options. Legacy group-128 files mislabeled as Q2_0 receive migration guidance. Fixed non-MSVC block declarations and GCC metadata-reader linking, with C ABI and ternary round-trip regressions (issue #184).
Added an AVX2 CPU vec-dot kernel for serialized group-64 Q2_0 weights against Q8_0, replacing the scalar x86 fallback for CPU expert layers. This weight format remains distinct from Bee's cache-facing GGML_TYPE_Q2_0S.
Enabled Qwen4Exp CUDA tensor splitting with shared MTP sidecars. HyperConnection initialization stays in the first layer's graph split, shared drafts omit unused trunk tensors, draft mixer and projected-state shapes are corrected, and mirrored output projections and padded empty logits keep backend sampling compatible with multi-GPU placement.
Fixed single-head tensor splits for Gemma KV caches. Whole-head ownership survives reshape, view, cache rotation, and output projection, and trailing host cache views close Meta subgraphs correctly, preventing tensor-split crashes. Added numerical and graph-boundary regressions.
Improved multi-stream KVarN decode routing and stores. Native-attention admission uses each stream's query width rather than the flattened batch size, qualified one-row multi-stream SWA decode consumes compressed records directly on CPU and CUDA, and other backends or wider SWA query batches retain materialization. CUDA descriptors use stream-local rings and read indices with global stage addressing, in-flight tails resolve the correct staging slots, and independent headwide decode stores run in parallel after validating actual stream ownership, with a serial fallback for shared destinations. Regression coverage includes wrapped rings, exact tails, and hinted stores.
Fixed self-contained KVarN state for per-sequence and single-sequence fixed stage layouts. Saves include every sealed compressed record independently of the F16 rollback window, preserving both fresh/cross-slot continuation and suffix rollback that reopens a completed group. The compact read-plan path and KVarN state format are unchanged; regressions cover restore into fresh or overwritten sequences, hard memory clears, and rollback replay.
Preserved historical slot checkpoints across disk save/restore and server restarts. An optional versioned appendix stores target, owned draft, and speculative checkpoint state, retaining Bee's atomic restore transaction and cross-slot reuse instead of probing compatibility by mutating live draft state. Blob lengths and metadata are validated before admission, malformed or truncated appendices are ignored without accepting partial checkpoint lists, incomplete writes report failure, UTF-8 paths remain supported, and save/restore byte counts include the appendix. Added F16/KVarN hybrid and owned-MTP checkpoint regressions.
Windows release builds now tolerate taskkill races when compiler children exit during shutdown, preserving the rolling CUDA build cache.

macOS:

macOS Apple Silicon

Linux:

Windows:

Docker:

CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.8
CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.8
CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.8
CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.8
ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.8
Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.8
SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.8

Browse all container images