Ollama apps now work with kapitsa serve, and Hermes, Aider and Goose launch in one command. With greedy decoding, MoE models run about 8% faster per step on M5 Macs, Q8_0 files are read with about 70% less CPU time, and an idle server gives its memory back.
A cancelled prompt keeps its saved checkpoints, so a retry doesn't read it all again; a stuck cancel restarts the engine instead of hanging the server; and broken model files are rejected. Swift apps can use Kapitsa through a package (build libkapitsa_sdk with CMake first).
Thanks to @chainhelen and @KeyingSong0123.
What's Changed
- DFlash2: include Q4 weight codes in the low-rank factor cache key by @chainhelen in #980
- bench/coding10: ten coding prompts for small models; little-coder's tool loop tuned to 30/30 at 1.0 s (4B) and 2.0 s (9B) per prompt (#350) by @alish-nace-23 in #984
- model: reduce 4B packed interleave scratch from 323 MiB to 171 KiB by @chainhelen in #979
- DFlash2: reserve packed projections and low-rank buffers in the memory plan by @chainhelen in #981
- KFlash names its own extensions: NI_KFLASH_ knobs with NI_DFLASH_ aliases, naming and local-state rules (#362) by @alish-nace-23 in #985
- Drex Max prefill on the M5 Pro: GEMMs over 2048 rows run as 2048-row dispatches, 16K request 9.6% faster, 4K 5.0%, readouts bitwise identical by @alish-nace-23 in #988
- kapitsa import attaches the matching catalog drafter; 16-row block needs a 4-bit draft head (#364, #967) by @alish-nace-23 in #987
- KFlash: online selector update on by default (lr 0.003) for the 35B-A3B on an M5 Pro (#362) by @alish-nace-23 in #989
- KFlash: NEON affine quantization cuts CPU time 5–10% on 4B matrices (M3 Ultra) by @chainhelen in #996
- KFlash: shrink low-rank scratch; 4B r=768 drafter load time -7.4% on M3 Ultra by @chainhelen in #997
- Docs: defaults follow the concept, not the measured cell (#368) by @alish-nace-23 in #999
- Safetensors: validate shape and byte ranges before deferred reads by @chainhelen in #998
- Docs: two leftovers of the old defaults rule (#368) by @alish-nace-23 in #1004
- kapitsa doctor --print-profile prints the chip class and the default table by @alish-nace-23 in #1000
- Move the duplicated chip checks into the profile predicates by @alish-nace-23 in #1001
- Docs: reword the 'because it was measured' reasons the first search missed (#368) by @alish-nace-23 in #1005
- The duplicated apple10 and M1 checks become two predicates: 62 inline chip checks to 15, same behavior (not verified on a GPU) by @alish-nace-23 in #1007
- Docs: the list and A/B gate wording the meaning search missed, plus an allowlist search (#368) by @alish-nace-23 in #1008
- The M5 Max and M5 Pro name gates become the wide and matrix classes: 4 inline chip checks left (not verified on a GPU) by @alish-nace-23 in #1009
- KFlash: hoist draft-row coefficients, reducing CPU dequantization time about 17% by @chainhelen in #1003
- The profile table lists the five class defaults that had no row (not verified on a GPU) by @alish-nace-23 in #1011
- Docs and lint: the last measured-here reasons, and a CI check that stops them coming back (#368) by @alish-nace-23 in #1012
- model: contiguous Q4 packing cuts CPU pack time 39–46% on 4B matrices by @chainhelen in #1002
- Say what is true about Clover, key the comment-gate allowlist on path (#368) by @alish-nace-23 in #1013
- Safetensors: reject unsupported selected dtypes before payload reads by @chainhelen in #1006
- The comment-gate check covers the whole repo and the 'not run, so' form (#368) by @alish-nace-23 in #1016
- Document the six name predicates as guard rows by @alish-nace-23 in #1014
- Fast profile (--profile fast): TATTN hi+lo (#818) on Flash-Next on Macs with matrix units, prefill 8-10% faster at 32K-64K, 4 of 72 flips vs Splash's 16 by @alish-nace-23 in #986
- Vision: read unpadded BF16 weights directly into destination by @chainhelen in #1018
- KFlash: vectorize CPU affine range scan (45–49% less quantizer time) by @chainhelen in #1017
- Fast profile on #986's merged build: Flash-Next decode the same at 32K and 3% faster per cycle at 64K, prefill 10% and 7% faster; TATTN clean under shader validation by @alish-nace-23 in #1021
- The seven class-gated defaults read their device profile row; defaults unchanged (#369) by @alish-nace-23 in #1015
- kapitsa serve --idle-unload: an idle server gives its weights back (#1020) by @alish-nace-23 in #1024
- AGENTS.md: an env-switch A/B through kapitsa bench needs its own server (bench strips NI_ variables; #372 ran A/A) by @alish-nace-23 in #1035
- Flash-Next: reject truncated mapped n-gram tensors by @chainhelen in #1019
- Lightning 8-bit attention projections (NI_NH_ATTN_Q8) measured: decode about the same, 4K prefill 8% slower on the M5 Pro; not a fast-profile member by @alish-nace-23 in #1036
- Qwen: preserve nameless tool calls as text (Not verified on a GPU) by @chainhelen in #1037
- Flash-Next: decode affine Q4 n-gram rows with NEON by @chainhelen in #1022
- The remaining chip predicates are rows in the profile table (#369) by @alish-nace-23 in #1038
- bench/coding10: same-leg rerun of opt0, lc, lc_best (tool loop ~5x faster on 4B, ~3.5x on 9B) by @alish-nace-23 in #1039
- qwen35_v8: default moe_emu_mode to 2 on Apple10 (moe8e MMA-exact passes for greedy verifies) by @gnanasreekar in #1028
- Profile rows cite the fused-merge break, not a missing rerun (#369) by @alish-nace-23 in #1040
- Benchmark: Mac Studio (2026) · M5 Ultra · 96 GB · Qwen3.8 Flash-Next 180B-A7B 4-bit by @gnanasreekar in #1042
- Benchmark: Mac Studio (2026) · M5 Ultra · 96 GB · Qwen3.6 35B-A3B 4-bit by @gnanasreekar in #1025
- engine: keep split stop strings out of replies by @chainhelen in #1043
- Five model loaders share one config reader (Not verified on a GPU) by @alish-nace-23 in #1044
- Four Qwen files share one env_int (Not verified on a GPU) by @alish-nace-23 in #1045
- LiliCorr and DSpark share one weight loader (Not verified on a GPU) by @alish-nace-23 in #1046
- Four bench tools share one Metal archive reader (Not verified on a GPU) by @alish-nace-23 in #1047
- Benchmark: MacBook Pro (16-inch, 2026) · M5 Pro · 48 GB · Qwen3.8 27B 4-bit by @optrv in #1026
- Qwen: reject malformed affine MTP sidecars by @chainhelen in #1041
- parser: preserve blank-name JSON tool calls as text (Not verified on a GPU) by @chainhelen in #1048
- ane: fast prefill on built-in-share non-matrix Macs (9B loaded TTFT -61.94%) by @KeyingSong0123 in #775
- engine: retain completed prefixes after prefill cancellation by @chainhelen in #1051
- Benchmark: Mac Studio (2023) · M2 Ultra · 64 GB · Qwen3.8 27B 6-bit by @mwheeler1982 in #1027
- Expert store: count the turn's stats before the go releases the caller (Not verified on a GPU) by @alish-nace-23 in #1055
- Benchmark: Mac Studio (2026) · M5 Ultra · 96 GB · Qwen3.5 4B 4-bit by @gnanasreekar in #1029
- Benchmark: Mac Studio (2026) · M5 Ultra · 96 GB · Qwen3.5 9B 4-bit by @gnanasreekar in #1030
- Preserve NaNs in BF16 conversion by @chainhelen in #1054
- Dory: share split weights in a separate SIMD reduction kernel by @chainhelen in #1052
- Benchmark: Mac Studio (2026) · M5 Ultra · 96 GB · Qwen3.8 27B 4-bit by @gnanasreekar in #1031
- Benchmark: Mac Studio (2026) · M5 Ultra · 96 GB · Nemotron 3.5 Lightning 30B-A3B nvfp4 by @gnanasreekar in #1032
- Benchmark: Mac Studio (2026) · M5 Ultra · 96 GB · Nemotron 3.5 Bolt 74B-A9B NVIDIA 24 GB by @gnanasreekar in #1033
- Benchmark: Mac Studio (2026) · M5 Ultra · 96 GB · Qwen3.6 35B-A3B 8-bit by @gnanasreekar in #1034
- Benchmark: MacBook Pro (16-inch, 2021) · M1 Max · 64 GB · Qwen3.8 27B 4-bit by @gxp-moyama in #1059
- Benchmark: MacBook Pro (16-inch, 2021) · M1 Max · 64 GB · Qwen3.5 4B 4-bit by @gxp-moyama in #1060
- Native engine: restart after cancellation timeout by @chainhelen in #1058
- GGUF: reject wrapped tensor and read ranges by @chainhelen in #1061
- Benchmark: Mac Studio (2026) · M5 Ultra · 96 GB · Nemotron 3.5 Super-VL 120B-A12B nvfp4 by @gnanasreekar in #1062
- Flash-Next: reuse router maxima (M4 kernel latency -3.9–7.5%) by @chainhelen in #1056
- Nemotron: FP8 CPU packing latency -86.1–89.7% by @chainhelen in #1063
- Nemotron: cut exact CPU Q8 quantizer latency 74–81% by @chainhelen in #1065
- README: a Japanese translation (README.ja.md), linked from the top of the English README by @itomatik in #1067
- Ngram: fix split GGUF row reads (Not verified on a GPU) by @chainhelen in #1066
- Agent launchers: hermes, aider, goose (#CON-293) by @alish-nace-23 in #1069
- Landing page and community leaderboard (#CON-290, #CON-303) by @alish-nace-23 in #1073
- packaging/homebrew: homebrew-core formula draft and status doc (CON-291) by @alish-nace-23 in #1070
- GGUF: cut exact IQ4_NL CPU row decode latency 58–79% by @chainhelen in #1076
- Swift package over existing C API (sdk/c/kapitsa.h) (#CON-301) by @alish-nace-23 in #1072
- Guide: Kapitsa as Xcode 26's chat model (#CON-294) by @alish-nace-23 in #1074
- Day-0 support playbook for new model releases (#CON-296) by @alish-nace-23 in #1075
- Ollama-compatible API: /api/chat, /api/generate, /api/tags and more (#CON-287) by @alish-nace-23 in #1071
- docs: link community leaderboard to kapitsa.vercel.app by @alish-nace-23 in #1078
- GGUF: Q8_0 CPU rows take 68.6-75.5% less time on ARM64 by @chainhelen in #1079
- Safetensors: reject wrapped relative read ranges (Not verified on a GPU) by @chainhelen in #1080
- GGUF: pack five-bit CPU codes with one write per word by @chainhelen in #1081
- M3 Pro 36 GB bench: 27B KFlash quick (38.1 tok/s) and 35B-A3B KFlash quick (111.3 tok/s) by @alish-nace-23 in #1083
- Attention: share K scratch only under shader validation (Not verified on M5 GPU) by @chainhelen in #1057
- Kapitsa 0.2.32 by @alish-nace-23 in #1085
New Contributors
Full Changelog: v0.2.31...v0.2.32