github nace-ai/kapitsa v0.2.27
Kapitsa 0.2.27
on GitHub

2 hours ago

What's Changed

  • release.yml: the Homebrew pour check reads brew's output without color, so v0.2.26's poured bottle no longer fails the step by @fearampgreed in #858
  • Splash mode: a model whose weights fit in free memory keeps Splash's own memory plan instead of a forced --expert-memory (35B-A3B, 23-30 GiB free: cold 4K TTFT -332 ms, decode +53.8 tok/s, n=3) by @alish-nace-23 in #867
  • vision: prepack Qwen weights (M4 Pro warm image encode time -2.5–3.6%) by @chainhelen in #868
  • NI_ANE_NONCAUSAL=1 (default off): Drex Max's noncausal pass splits its FFN with the Neural Engine in 2048-row slabs (M5 Pro, 4K latency x0.832, 16K x0.865, n=3; floor gate not run) by @alish-nace-23 in #874
  • Research: online drafter adaptation for Kapitsa, 87-method survey, top 3 and phased plan, calibration simulator (not verified on a GPU) by @alish-nace-23 in #871
  • Benchmark: MacBook Pro (16-inch, 2026) · M5 Pro · 48 GB · Qwen3.6 35B-A3B 5-bit by @valkorobov in #641
  • Benchmark: MacBook Pro (14-inch, 2026) · M5 Pro · 64 GB · Qwen3.8 Flash-Next 180B-A7B 2-bit experts by @stardude1 in #733
  • Benchmark: MacBook Pro (14-inch, 2026) · M5 Pro · 64 GB · Qwen3.6 35B-A3B 6-bit by @stardude1 in #734
  • Benchmark: MacBook Pro (14-inch, 2026) · M5 Pro · 64 GB · Qwen3.6 35B-A3B 2-bit experts by @stardude1 in #735
  • Benchmark: MacBook Pro (14-inch, 2026) · M5 Pro · 64 GB · Nemotron 3.5 Bolt 74B-A9B NVIDIA 24 GB by @stardude1 in #736
  • Benchmark: Mac mini (2026) · M5 Pro · 64 GB · Qwen3.6 35B-A3B 5-bit by @PGDAO in #800
  • Benchmark: MacBook Pro (16-inch, 2024) · M4 Pro · 48 GB · Qwen3.8 27B 6-bit by @BPHvZ in #851
  • Benchmark: MacBook Pro (16-inch, 2021) · M1 Max · 32 GB · Nemotron 3.5 Bolt 74B-A9B NVIDIA 24 GB by @tragdor85 in #833
  • Benchmark: Mac Studio (2026) · M5 Max · 128 GB · Qwen3.8 Flash-Next 180B-A7B 4-bit by @ezcoding in #869
  • Benchmark: MacBook Pro (14-inch, 2021) · M1 Max · 64 GB · Qwen3.6 35B-A3B 4-bit by @phaedrus1992 in #816
  • Benchmark: MacBook Pro (16-inch, 2021) · M1 Max · 32 GB · Qwen3.6 35B-A3B 4-bit by @tragdor85 in #832
  • bench/community/RESULTS.md: regenerate from results by @alish-nace-23 in #877
  • q20 converter: --gguf takes a split set (shard 2 of 2 no longer a KeyError) by @nocmt in #645
  • bench/community/RESULTS.md: regenerate from results by @alish-nace-23 in #879
  • Benchmark: MacBook Pro (16-inch, 2021) · M1 Max · 32 GB · Qwen3.8 27B 4-bit by @tragdor85 in #831
  • Benchmark: Mac Studio (2023) · M2 Ultra · 64 GB · Qwen3.8 27B 8-bit by @mwheeler1982 in #870
  • bench/community/RESULTS.md: regenerate from results by @alish-nace-23 in #883
  • Research tools: history-lane replay and draft-head coverage by language (CPU only, not verified on a GPU) by @alish-nace-23 in #872
  • NI_ANE_NONCAUSAL on by default at the plain split's share (16/34 units on the M5 Pro): Drex Max prefill 4K x0.72, 16K x0.79; readout moves <=6.9e-5, top answer unchanged on 8 inputs by @alish-nace-23 in #875
  • Benchmark: MacBook Pro (16-inch, 2021) · M1 Max · 32 GB · Clover-35B-A3B-Preview 4-bit by @brechtm in #881
  • Server “requests” table: swap column headers and “read/s” explanation by @sboukortt in #825
  • ci: the community results table is a merge gate, not an after-the-fact warning by @alish-nace-23 in #880
  • README context charts to each model's limit on the M5 Max (Lightning 5,138 → 823 prefill, 240 → 21 decode tok/s from 1K to 512K): the harness's long corpus, --sweep-task code and bench/results/2026-11-02-m5-max-long by @fearampgreed in #884
  • kv: an opt-in int8 KV cache for the Qwen3.5/3.6/3.8 and Flint attention (NI_KV_I8); Flash-Next/Clover not GPU-verified by @yangweijie in #838
  • tools/logit_floors: Nemotron 3.5 Lightning's, Super-VL's and Bolt's noise floors on the M5 Max (decode gates: mean KL 9.95e-5 / 5.41e-5 / 8.03e-5) by @fearampgreed in #885
  • Benchmark: Mac Studio (2026) · M5 Max · 64 GB · Qwen3.8 27B 4-bit by @Cromemadnd in #886
  • bench/community/RESULTS.md: regenerate from results by @alish-nace-23 in #889
  • Nemotron's verify attention on the M5's matrix units with the context split (nh_attn_t_part): Lightning decodes 3.8-4.4x faster at 255K tokens (41 → 167 tok/s), Super-VL and Bolt 1.9-2.4x at 133K by @fearampgreed in #887
  • Nemotron's prefill attention with its K/V tiles staged for 8 tokens (nh_attn_ts_s8), bit for bit the same output: Lightning's 512K-token prefill 580 → 414 s (1.40x), Super-VL and Bolt 1.2-1.3x at 133K by @fearampgreed in #890
  • tools/logit_floors: the Qwen3.6 35B-A3B's and Clover's noise floors on the M5 Max (decode gates: mean KL 3.37e-4 / 8.87e-4) by @fearampgreed in #891
  • The 35B's and Clover's tree verify attention on the M5's matrix units (attn8t_partial_zone_g8): 35B decode 1.3-1.5x faster at 131K-249K tokens (97 → 135 tok/s at 249K) by @fearampgreed in #892
  • README context charts after the matrix-unit attention (#887, #890, #892): Lightning decodes 116 tok/s at 512K tokens (was 21), Bolt 91 at 255K (was 31), the 35B 114 at 259K (was 80) by @fearampgreed in #894
  • tools/logit_floors: Qwen3.8 27B's noise floor on the M5 Max (decode gate mean KL 4.88e-7, general gate 1.18e-4) by @fearampgreed in #895
  • tools: fail on DFlash2 tensor regressions with a bytewise checker by @KeyingSong0123 in #893
  • The 27B's wide tree verify attention on the M5's matrix units (attn8t_partial_tree_g6, P to fp32 accuracy): a 16-row verify cycle 25% cheaper at 131K tokens; identical temperature-0 continuations by @fearampgreed in #897
  • Dory's verify attention in one threadgroup per chunk and KV head for all rows (dory_attn_partial_rows), bit for bit the same: a verify cycle 9% cheaper at 133K tokens by @fearampgreed in #898
  • metal: prepack 4B QMM on M4 (decode +7.7% short, +5.3% 4K) by @KeyingSong0123 in #896
  • logit_dump --verify-rows 16 / 32, the wide tree verify: the 27B's attn8t_partial_tree_g6 (#897) passes its M5 floor's gate there (KL mean 1.7e-7, p99 2.2e-6 at 16 rows); amends #897's results README with that gate and its contention by @fearampgreed in #901
  • erratum: the contention notes of #890's and #892's results READMEs from their own run JSONs (the desktop app held 12-27% and 38-58% of the GPU, not ~10%), and #892's kernel-alone timings marked as unrecorded by @fearampgreed in #902
  • README context charts: the 27B and Dory on main after #897 and #898, four requests a size (the 27B decodes 32.6 tok/s at 256K, was 27.6; 40.9 at 128K, was 34.2) by @fearampgreed in #900
  • Research doc: Weaver is 2.03x over no speculator and 1.15x over its own chain on the 27B (M5 Pro, 108 legs), no clear gain in Kazakh (docs only) by @alish-nace-23 in #905
  • Dory's prefill attention with the head dim split over simdgroup pairs (dory_attn_td2_g4s8): a 255K-token prompt's first token 586.7 -> 374.7 s, 133K 185 -> 125 s on the M5 Max (other M5s untested); the attention 1.8-1.9x faster, its sum over the head dim reassociated by @fearampgreed in #904
  • README context charts: Dory after its prefill attention moved to simdgroup pairs (#904): prefill 407 -> 700 tok/s at 256K, 695 -> 1,102 at 128K by @fearampgreed in #907
  • Flint's verify attention on the M5's matrix units (flint_attn_t_rows): an 8-row pass 11.0 -> 1.0 ms a layer at 262K positions; prompt lookup 2.2-2.5x faster at 123K-236K tokens, the same text by @fearampgreed in #908
  • perf: retain MoE top-k results per lane (M4 Pro prefill routing -26–43%) by @chainhelen in #903
  • Dory's decode attention with a row's keys 8 at a time (dory_attn_partial_k8 / _rows_k8): 7.6-10% less a decode cycle at 134K-259K tokens on the M5 Max, identical temperature-0 text; moves outputs (verify KL 8.4e-7), failing Dory's zero decode gate by @fearampgreed in #906
  • Dory's self-drafts read a 4K window in their output layers' full attention (NI_DORY_DRAFT_WIN): decode 3.5-3.7% faster at 134K-259K tokens on the M5 Max, the same text by @fearampgreed in #909
  • Flint drafts with itself from 180K-token prompts (MagicDec: 2 drafts on a 4K KV window, one verify pass): decode +8-10% at 187K and +8-28% at 236K tokens on the M5 Max; moves outputs: step against verify fails Flint's M5 floor on top-1 (0.9805 over 768 rows), KL within it by @fearampgreed in #910
  • The 35B's tree verify attention with its K / V tiles staged once a threadgroup (attn8ts_partial_zone_g8), bitwise the same: decode +9.6% at 132K and +11.2% at 254K tokens on the M5 Max by @fearampgreed in #911
  • Research doc: rank the online drafter plan for English code first: history lane +13.3% and selector update +8.6% tokens per cycle on the English agent session, head work optional (docs only) by @alish-nace-23 in #912
  • The 27B's 8-row tree verify on its wide verify's matrix-unit kernel (attn8t_partial_tree_g6) instead of the zone's ALU kernel: decode +32.5% at 132K and +44.9% at 254K tokens on a 2-slot server, M5 Max by @fearampgreed in #916
  • draft: key the read-ahead by prompt_cache_key, not the reply text by @alish-nace-23 in #914
  • The 27B's drafting block chosen per request: 8 rows from 16K-token prompts, 16 below (DFlash2::set_block): decode +7.1% at 32K and +25.0% at 131K tokens on a one-slot server, M5 Max, the same text by @fearampgreed in #917
  • Benchmark: Mac Studio (2026) · M5 Max · 48 GB · Qwen3.8 27B 4-bit by @reagle in #918
  • Benchmark: Mac Studio (2026) · M5 Max · 48 GB · Qwen3.6 35B-A3B 4-bit by @reagle in #919
  • README context charts: the 35B and the 27B on main after #911, #916 and #917, four requests a size (the 27B decodes 42.5 tok/s at 256K, was 32.6; the 35B 136, was 114) by @fearampgreed in #920
  • Flash-Next's folded gated residual on wider grids (hc_down_sk4, hc_up_mix4_c2), bitwise the same: decode 105.6 -> 107.5 tok/s (21.92 -> 21.45 ms of GPU a cycle) on the M5 Max by @fearampgreed in #921
  • Flash-Next's wider folded-residual grids (#921) re-measured on an idle GPU: decode 105.9 -> 108.1 tok/s (21.84 -> 21.40 ms of GPU a cycle) on the M5 Max (docs and data only) by @fearampgreed in #927
  • Results: Qwen decode probes that did not pay on the M5 Max (Flash-Next qmvs_multi variants 5-48% slower, MTP depth 3 102.9 vs 108.05 tok/s, a repacked qmm8t layout 4-8% on loads alone; results only) by @fearampgreed in #928
  • flint_bench fails on a missing or empty bench/corpus.txt instead of looping forever (the prompt it builds is the same) by @fearampgreed in #930
  • The verify MoE's distinct experts found by a threadgroup instead of one thread (moe4_unique), bitwise the same: Flash-Next decode 108.5 -> 111.2 tok/s (21.32 -> 20.78 ms of GPU a cycle), Flint's 8-row verify 16.69 -> 16.33 ms on the M5 Max by @fearampgreed in #922
  • feat(api): report reasoning tokens in chat completions usage by @gnanasreekar in #925
  • Flash-Next's verify MoE reads its rows' inputs through the caches instead of staging them (moe4_gate_up_d, moe4_down_d), bitwise the same: decode 108.2 -> 109.3 tok/s (21.37 -> 21.16 ms of GPU a cycle) on the M5 Max by @fearampgreed in #923
  • EXL3 prefill: decode weights into MMA registers, cutting expert chain GPU time 4.3–7.7% on M4 Pro by @chainhelen in #926
  • README: Flash-Next's M5 Max cell re-measured on main af7d795 after #921-#923: greedy decode 107.3 -> 113.9 tok/s (+6.2%), 4K prefill 1,701 -> 1,726, T 0.6 108.3 -> 108.5 (docs and data only) by @fearampgreed in #936
  • Kapitsa 0.2.27: sets the version in CMakeLists.txt, the KAPITSA_VERSION fallbacks in src/, install.sh's example and packaging/homebrew/kapitsa.rb, as #856 did for 0.2.26. The tag v0.2.27 goes on the merge commit. by @fearampgreed in #937

New Contributors

Full Changelog: v0.2.26...v0.2.27

Check out latest releases or
releases around nace-ai/kapitsa v0.2.27

Don't miss a new kapitsa release

NewReleases is sending notifications on new releases.

Get notifications