- MiMo-V2.6-Flash: sushi's second model.
MiMo-V2.6-Flash-Sushi-2.3bpwserves text and image input from one
resident pack with native MTP, up to its full 1M-token context on a 128 GB Mac. - Sushi packs stream from SSD:
--ssd-budget-gb Nkeeps the trunk resident and streams the routed experts from
SSD, so a Mac with less memory than the pack can serve it; replies are identical to a resident load. Every Sushi
Qwen pack and MiMo-V2.6-Flash-Sushi-2.3bpw stream, and Sushi-2bpw serves on a 32 GB M1 Max at a 20 GB budget. - Faster Flash-Next: on an M5 Max, Sushi-4bpw decodes 8% faster (83 -> 89 tok/s) and prefills a 10k-token prompt
19% faster (1,796 -> 2,139 tok/s). oMLX's tensor-unit sparse attention now serves prefill from the first sparse
chunk, GDN prefill runs a software-pipelined recurrence, batched decode overlaps GPU work with graph building, and
--prefill-decode-sharekeeps decoders moving while a long prompt prefills. Thanks @STRML and @cowboycoderhq. - Better and smaller packs: new expert weights for Sushi-4bpw (KLD 0.0632 -> 0.0592) and Sushi-3bpw
(0.1047 -> 0.1036), and Sushi-2bpw for 48 GB Macs. - Live sessions on
/metrics.json: every in-flight request and every cached conversation, with its phase,
context against the model's limit and the GPU memory its KV holds. Thanks @yoyo930021 and @ddalcu. - Updates itself:
sushi updateinstalls the newest release after checking its SHA-256 and signature and keeps
the old one for--rollback; a daily check, one-click update from the chat page, orbrew install beamivalice/tap/sushi. - Fixes: MiMo long-context memory returns to the OS, SSD prompt-cache restores keep one copy, JSON-constrained
logprobs pair with their tokens, and a failed model load can be retried after a rescan. Thanks @brandondyal and
@jasontitus; the EXL3 engine is now a module mlx-serve builds against, thanks @ddalcu.