JustVugg/colibri v1.5.0

v1.5.0 — DeepSeek V4 Flash, and eight advisories published

on GitHub

51 pull requests from 15 contributors.

A fifth engine: DeepSeek V4 Flash

@DrewZt's #165 brought the target-only CPU engine — MLA + DSA sparse attention, 43 layers, 256 routed experts plus one shared, top-6. The official checkpoint streams with no conversion: routed experts stay native fp4, dense stays fp8-e4m3 with UE8M0 block scales.

#839 then made it usable on the hardware most people own. The rows16 fp4 fast path was gated on __AVX512F__ || __aarch64__, and Intel dropped AVX-512 from consumer parts at Alder Lake — so on most laptops and desktops sold since 2021 every routed expert fell back to the scalar path. Measured: 601 s for 8 tokens. The release adds the AVX2 backend, an fp8 rows8 dense path, O_DIRECT expert reads, .coli_usage warm pins, and KV prefix reuse so a second chat turn stops re-prefilling the first.

Speculative drafting is implemented, verified, and off: 1 accepted candidate in 15 for the markov drafter, 10 in 24 for full MTP, and one 14-token answer that took 495 seconds replaying its rejected suffixes. The code stays with the numbers beside it.

Eight security advisories, published with this release

Fixed in #841 and disclosed today rather than patched quietly — the fix commits were already public, and silence would only have kept users on 1.4.0 from learning they should upgrade.

• load_scalar() wrote a file-sized tensor into a 4-byte stack slot; RSP was fully controlled in the reported crash
• load_cfg() indexed layer_types[] by layer count instead of array length — a crafted config.json alone, no weights
• SERVE max_tok accepted negatives, which inverted the context check and overflowed the KV cache
• All-NaN router logits left the top-k pick at -1 and used it as an index; release builds did not fault, they returned wrong numbers over corrupted memory. colibri.c had been guarded since test_logit_nan.c; the three sibling engines never received it
• /profile served per-turn telemetry to unauthenticated callers even with --api-key set
• Unbounded threads and a renewable timeout let a slowloris drip pin a thread forever

Reported by @zh-Processor (six), @aeonframework and @ajmeese7 (two, already closed by #413 and re-verified here). All credited on the advisories.

Everywhere else

Windows gained native HIP with fail-closed runtime binding (#788, @Kenneth-Javier), validated on physical gfx1151. Apple Silicon: METAL=1 builds on a stock macOS again (#807), and Inkling's shared experts reach the GPU for 1.75 → 2.46 tok/s (#757, @rgbkrk). CUDA got fmt=8 dense and expert-group kernels (#817, @kreuzzelg) and a paged ragged KV runtime (#795).

OMP_NUM_THREADS is now sized from physical cores on every platform (#805, @ThefloorMiner) — and, after this release exposed that it reached only GLM, for all five engines. A one-iteration OpenMP region is gone (#806), measured on a 744B model across three runs per side by the reporter of the counter-evidence.

Kimi K3 learns from use (.coli_usage, autopin, decay), the adaptive LRU survives autopin (#815), the stateful KV tail at the NGEN limit is fixed (#567), and the CLI banner finally names the model you actually loaded instead of GLM-5.2 (#832).

Thanks

@ZacharyZcR (10 PRs), @ThefloorMiner (7), @terrizoaguimor, @rgbkrk, @mgua, @kreuzzelg, @bherald, @DrewZt, @winklemad, @monotophic, @dyKiU, @anrasi, @RDouglasSharp, @Kenneth-Javier — and @zh-Processor, @aeonframework, @ajmeese7 for the security work.

Special mention to @ThefloorMiner, who answered a performance claim by running the real 744B model with .coli_usage snapshotted between runs, three runs per side and a byte-for-byte output diff — and reported numbers that contradicted the original framing. That is the standard this project wants.