What's Changed
- README: the 27B's and 35B-A3B's M5 Max cells on main 87d7dd5 (#964, #969, #970): 27B prefill 1,104 -> 1,126 tok/s at 4K and 1,032 -> 1,070 at 16K, past Splash at both; 35B-A3B 6,558 -> 6,619; decode inside the ranges (docs and data only) by @fearampgreed in #971
- M4 decode: packed GEMMs, lazy GDN fold, and a low-rank DFlash2 head, on the 10-core M4 (4B KFlash 144.8 to 152.3 tok/s) by @darkhanakh in #953
- Qwen: reuse X8 preparation across small-row projections by @chainhelen in #837
- The 35B-A3B's prefill as a two-half GPU wavefront in 8192-row chunks on the M5 Max (NI_PF_WAVE): one half's DeltaNet scan and attention beside the other half's expert GEMMs, the same logits as half-size chunks; 16K prefill -5.7% (2,776 -> 2,618 ms), cell 5,714 -> 6,061 tok/s, 4K level by @fearampgreed in #975
- Results: the chunked DeltaNet scan on staged register state (2.77 ms against the recurrence's 1.21 at the 35B-A3B's 2048 tokens) and its matrix products on the tensor units (0.54 ms alone), and the 27B's GPU FFN part beside the next mixer (-0.4 to -1.3%, inside the spread): none shipped (docs and data only) by @fearampgreed in #976
- Rules: a feature off by default on every model and Mac never merges; a measured win turns on where it won by @alish-nace-23 in #977
- metal: prepack 9B down projections on M4 Pro 48 GiB (screened decode +1.3–1.5%) by @chainhelen in #973
- README: the 35B-A3B's M5 Max cell on main 39f20c8 (#975, the GPU prefill wavefront): 16K prefill 5,718 -> 6,046 tok/s (+5.7%), 4K one chunk either way (6,539), decode 351 (docs and data only) by @fearampgreed in #978
- serve: read default arguments from serve.json for Homebrew services by @chainhelen in #974
- ane: keep the reason when the Neural Engine program fails to compile or load (the service queue's @catch (...) dropped a C++ exception's text) by @fearampgreed in #982
- DFlash2 history lane on by default for the 35B-A3B on an M5 Pro: about 13% faster, same outputs by @alish-nace-23 in #944
- Kapitsa 0.2.31: sets the version in CMakeLists.txt, the KAPITSA_VERSION fallbacks in src/, install.sh's example and packaging/homebrew/kapitsa.rb, as #972 did for 0.2.30. The tag v0.2.31 goes on the merge commit. by @fearampgreed in #983
New Contributors
- @darkhanakh made their first contribution in #953
Full Changelog: v0.2.30...v0.2.31