nace-ai/kapitsa v0.2.11

Kapitsa 0.2.11

on GitHub

Highlights

• Context: the default is the model's own limit, lowered by what memory holds (#373).
• 27B prefill on M5: attention on the tensor units (#381; serve cold 16K 1.18x, 4K 1.05x on the M5 Pro). NI_QATTN_G6=0 restores the old kernel.
• 35B-A3B prefill on M5: attention with the head dim split over a pair of simdgroups (#365; cold 16K 1.14x, 4K 1.04x). NI_QATTN_D2=0 restores the old kernel.
• Serving: a side request no longer evicts a conversation's slot and waits for its SSD copy (#380); a short suffix at the turn boundary takes the prefill path (#371); a turn's reasoning is put back when the client sent it without (#375); float tool arguments render as 1.5, not 1.500000 (#372).
• Prefill tok/s beside the time to the first token in the chat page, request log and kapitsa bench (--timing, #374).

Numbers are the authors' own; the release was cut without extra GPU runs.

What's Changed

• 35B prefill attention: the head dim split over a pair of simdgroups (qattn_t_g8d2s8; serve cold prefill 1.14x at 16K, 1.04x at 4K on the M5 Pro) by @alisher-amantay in #365
• bench/results: M3 Pro 36 GB, the ten catalog models against every engine on macOS 14 and 1K to 32K (Kapitsa fastest decode on 7 of 9 with a rival, 1.21-2.90x; Nano 4B / 9B v2 0.99x / 0.84x) by @alisher-amantay in #369
• README: M3 Pro 36 GB on the poster, Kapitsa's decode fastest on 8 of 10 models (1.2-2.9x; behind on both Nanos), headline 32 of 36 (after #369) by @alisher-amantay in #370
• Prefill tok/s beside the time to the first token (chat page, request log, bench; --timing) by @alisher-amantay in #374
• Engine: a suffix that holds the turn boundary and is under 25 rows takes the prefill path by @alisher-amantay in #371
• Engine: put back the reasoning of a turn the client sent without it by @alisher-amantay in #375
• Benchmark: MacBook Pro (14-inch, 2025) · M5 · 32 GB · Nemotron 3.5 Lightning 30B-A3B nvfp4 by @OpenClawdd in #377
• Benchmark: MacBook Pro (14-inch, 2025) · M5 · 32 GB · Recursive Flint 40B-A2.5B 4-bit by @OpenClawdd in #378
• README: M5 Pro 35B-A3B Kapitsa cell 2,729 / 214 -> 2,802 / 201 after #365 by @alisher-amantay in #376
• Chat templates: a float tool argument keeps its text (1.5, not 1.500000) by @alisher-amantay in #372
• Context: the default is the model's own limit, lowered by what memory holds by @alisher-amantay in #373
• 27B prefill attention on the M5's tensor units (qattn_t_g6d2s8; serve cold prefill 1.18x at 16K, 1.05x at 4K, turns after a 16K prefix 1.13-1.20x on the M5 Pro) by @alisher-amantay in #381
• Benchmark: MacBook Pro (14-inch, 2025) · M5 · 32 GB · Qwen3.5 9B 4-bit by @OpenClawdd in #383
• README: no multipliers in What's new (Flint: Kapitsa reads prompts faster than rec and decodes faster than llama.cpp) and no "no spec" label on the poster and tables by @alisher-amantay in #382
• Engine: a side request does not evict a conversation's slot; wait for its SSD copy by @alisher-amantay in #380
• Kapitsa 0.2.11 by @alisher-amantay in #387

New Contributors

• @OpenClawdd made their first contribution in #377

Full Changelog: v0.2.10...v0.2.11