github nace-ai/kapitsa v0.2.8
Kapitsa 0.2.8

latest releases: v0.2.35-rc1, v0.2.34, v0.2.33...
20 days ago

Cached follow-ups read their new prompt as verify passes (#315, #328): on a dense target that verifies 16 rows wide (the 27B), a cached turn's short prompt suffix now runs as verify passes instead of a prefill. The first token comes 60-300 ms sooner (16K follow-up TTFT 402 to 325 ms and 401 to 341 ms). The small 4B stays on the prefill path, where the pass would cost 30-40 ms. NI_VPF=1 forces it on, NI_VPF=0 off.

27B Neural Engine split on the M5 (#318): the GPU's share of the FFN split runs on the M5's matrix units. On the M5 Pro, cold prefill goes from 494 to 619 tok/s at 4K (1.25x) and 441 to 516 at 16K (1.17x).

Dory's prefill attention on the M5's matrix units (#314): 1.24x / 1.22x in the kernel's A/B; the 4B's 4K / 16K prefill reads 1,656 / 1,311 tok/s on the M5 Pro.

Flint on macOS 14 (#303) and IQ3_XXS / IQ1_S expert kernels (#311, no default change).

Docs, charts and benchmark results (#313, #316, #317, #319-#321, #323).

Don't miss a new kapitsa release

NewReleases is sending notifications on new releases.