Warning
Rolling preview build from v0.5.0 at d51f2f5e0405. Assets and moving Docker tags are replaced by newer successful branch builds. Last updated: 10/11/2026 17:52.
Changelog
- Updated the llama.cpp base from
d327bf1f0to3c809f443and ggml to 0.28.0. Inherited changes include DeepSeek V4.1, LiquidAI LFM3/d1 text/audio/vision models, Limite 1B, Mistral Large 4, OCP FP8 E4M3 weights, batched tensor transfers, no-allocation memory measurement, realtime conversations, model memory scheduling, and backend/server/Web UI updates. - Added opt-in SINQ input-column preconditioning for original floating-point Qwen3.5 models via
llama-quantize --sinq-config FILE, using existing Q3_K/Q4_K, Q5_0/Q5_1, Q6_K/Q8_0 and IQ1/IQ2/IQ3/IQ4 codecs and CPU/CUDA kernels. Per-tensor settings control width, iterations and strength, with tensor/layer/role overrides, imatrix compensation and--sinq-diagnostics-jsonreports. Embeddings, output heads and auxiliary projections retain ordinary quantization. See the SINQ guide. - Added an optional offline SINQ optimizer to plan, generate, score, select, assemble and validate mixed stock/SINQ/AWQ candidates. It supports locked assignments, local/blockwise and Fisher scoring, and bounded groupwise KLD selection with optional surrogate filtering. Verified artifacts and source/stock/corpus locks bind results to inputs; generation and selection support resume and explicit thread/resource limits. Activation export verifies matching Qwen3.5 checkpoints. Advanced scoring supports CPU/CUDA and FP32/BF16 without silent reference rounding or backend fallback. Selection scores do not establish held-out quality gains.
- Added v2 SINQ GGUF metadata and FP16 scale companions for CPU and qualified CUDA inference, with shared runtime scales and reusable scaled inputs. LoRA retains original inputs. Loading validates companions and rejects unsupported backends, row/tensor splits and Hadamard combinations; conditioned sources cannot be copied or requantized. These GGUFs require BeeLlama: other engines may recognize the standard weight IDs but ignore companions and produce incorrect output.
- Added
llama-perplexity --ppl-fulland--kld-token-outputfor all N-1 corpus targets, including remainders and context boundaries, with per-context state resets. BF16 comparisons report full-vocabularyKL(P_BF16 || P_candidate), distribution statistics and per-token NLL/top-1 TSV. Reference reads and scoring overlap decoding; scoring respects thread limits, and output validation protects input paths. Measure the compact-reference floor with a matched BF16 self-comparison; keep context, batch, ubatch and runtime settings matched. See the argument reference. - Integrated FP8 GGUF support while preserving Bee's low-bit cache/state IDs and Prism PQ2_0/PTQ1_0 IDs. FP8 wire ID 43 maps to Bee's internal type; Q6_0 uses private wire ID 144. Added external-file, writer-ID, ABI and roundtrip regressions.
- Adapted KVarN and precision-tail allocation, accounting and exact fitting to upstream's model-owned no-allocation mode, including ISWA subcaches. Host checkpoints combine native strided reads with batched irregular slices, preserving serialized layout and transactional restore ordering, cancellation and atomic publication.
- Fixed Qwen4Exp multi-stream QSA masks and pooled-indexer shapes with KVarN/exact tails. Shared MTP sidecars borrow target embedding/output weights while retaining owned hybrid/indexer memory; dense legacy heads skip unused indexer uploads. Conversion accepts
model.mtp-prefixed HyperConnection mixers, preserving standalone-head names and PLE filtering. Added missing-weight, QSA/dense-head decode and standard/KVarN draft replay regressions.
macOS:
Linux:
- Ubuntu x64 CPU
- Ubuntu arm64 CPU
- Ubuntu x64 CUDA 12.4
- Ubuntu x64 CUDA 13.3
- Ubuntu x64 Vulkan
- Ubuntu x64 ROCm 7.2
- Ubuntu x64 SYCL
Windows:
- Windows x64 CPU
- Windows x64 Vulkan
- Windows x64 SYCL
- Windows x64 CUDA 12.4 - DLLs
- Windows x64 CUDA 13.3 - DLLs
- Windows x64 HIP
Docker:
- CPU:
docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-preview-v0.5.0 - CUDA:
docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-preview-v0.5.0 - CUDA 12:
docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-preview-v0.5.0 - CUDA 13.3:
docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-preview-v0.5.0 - ROCm:
docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-preview-v0.5.0 - Vulkan:
docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-preview-v0.5.0 - SYCL:
docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-preview-v0.5.0