Anbeeld/beellama.cpp v0.4.0 on GitHub

Changelog

Updated the llama.cpp base through upstream commit 32e789fdf. This includes EAGLE3 and expanded MTP support; new multimodal models and video input; LFM2/LFM2.5 tool parsing; router, API, and Web UI updates; the ET-SOC backend; and updates across the existing compute backends.
Added KVarN target-context KV-cache compression through --cache-type-k and --cache-type-v values kvarn2, kvarn3, kvarn4, kvarn5, kvarn6, and kvarn8. It supports independent K/V widths, Qwen3.6 and Gemma 4, unified and non-unified caches, separate SWA cache types, CPU/CUDA stores, and native CUDA FlashAttention. Draft and auxiliary contexts continue to use standard cache types; unsupported placements fail closed or use an explicit bit-width-matched fallback.
The standard KV-cache surface now includes q2_0, q2_1, q3_0, q3_1, and q6_1 alongside q6_0, with CPU/CUDA quantization, SET_ROWS/GET_ROWS, CUDA MMQ/vec-dot, and FlashAttention vector coverage. Cache-facing q2_0 uses GGML_TYPE_Q2_0S, keeping it distinct from upstream's serialized Q2_0 weight format.
New KV cache precision tail (KVCPT) controls --kv-tail-tokens and --kv-tail-type keep the newest attention-visible entries exact in F16 or BF16 for standard quantized and KVarN target caches. Partial tails use compact overlays and fully covered groups can use native exact storage. Tail routing respects attention bias and cache placement, while state save/restore, prompt-cache reuse, sequence operations, and server handoff preserve the same model. Restores are transactional, and incompatible v0.3.1 sessions fail closed.
Scoped --cache-ram to prompt-cache storage instead of active context checkpoint policy. Prompt-cache saves are byte-counted against the host budget before commit, KV rollback aligns to multimodal chunk boundaries, text-only slot save/restore works when an mtmd projector is loaded, and MTP prompt-cache reuse no longer requires regular checkpoints.
Replaced the fork DFlash implementation with upstream draft-dflash. The speculative type uses upstream's draft-dflash name, and draft GGUFs must use upstream's dflash architecture, metadata keys, tensor names, and tokenizer contract. Bee retains a default-on profit-based draft-depth controller and reasoning-loop guard while using upstream token, sampler, and checkpoint behavior. Without --spec-draft-n-max, the draft limit comes from dflash.block_size - 1; an explicit value takes precedence.
Removed TurboQuant/TCQ cache formats, TQ3/TQ4 weight formats, DDTree, CopySpec, the fork DFlash ring/capture/tape and reduced-verifier paths, the fringe controller, and their private arguments and environment variables. The turbo2, turbo3, and turbo4 cache names, including TCQ variants, warn and redirect by width to KVarN for target caches or a standard low-bit type for draft caches. Legacy TQ GGUF type IDs fail with a re-quantization error; copyspec, suffix, and recycle fail with migration guidance to draft-dflash or upstream n-gram modes.
Replaced the CUDA FlashAttention quant build policy with 50 standard vector pairs and 15 balanced KVarN fast-decode pairs by default. Standard quant pairs follow the KVarN bit-pair rules; homogeneous F16 and BF16 pairs retain native precision-tail support. GGML_CUDA_FA_ALL_QUANTS=ON expands these to 169 standard pairs and all 36 ordered KVarN pairs. The default-on GGML_CUDA_KVARN option is the single KVarN compilation gate for CUDA and HIP; disabling it omits the dedicated kernels and template instances.
Hardened server control and router APIs. /v1/chat/completions/control reports success only when an active reasoning sampler accepts the transition. GET /models is read-only and sanitized; authenticated refresh uses POST /models/reload. Hugging Face tokens reach child processes only through HF_TOKEN and are excluded from arguments, presets, logs, and responses.
Updated release packaging with CUDA 12.4/13.1 assets, upstream-DFlash and KVarN container metadata, ROCm/HIP shuffle compatibility, Windows CPU OpenMP runtime packaging, and updated Intel SYCL images. Every platform build, package, container, and publication step uses the same resolved source SHA; publication stops if the selected branch or tag moves.

macOS:

macOS Apple Silicon

Linux:

Windows:

Docker:

CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.0
CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.0
CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.0
CUDA 13: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.0
ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.0
Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.0
SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.0

Browse all container images