v1.7.2 — Memory you can count on, faster concurrent Qwen
Bugs fixed in this release, by severity and area:
| Severity | Memory & admission | Server & API | Models & loads | Engine & tests | Total |
|---|---|---|---|---|---|
| High | 21 | 10 | 2 | 2 | 35 |
| Medium | 21 | 16 | 8 | 4 | 49 |
| Low | 17 | 21 | 7 | 6 | 51 |
| Total | 59 | 47 | 17 | 12 | 135 |
- The context sushi names is the context it admits: requests are checked against the GPU limit macOS declares
(iogpu.wired_limit_mb) less one safety margin, as in 1.6.0, not against free RAM that other apps move. A 64 GB Mac
at the recommended limit serves Qwen3.8-Flash-Next STQ 2.75bpw at a 262K context with kv8 and images. A request that
fits once the running ones finish now waits its turn instead of being refused, and the margin is a setting
(--wired-margin-gib, Settings › GPU memory). - Concurrent Qwen requests are faster on Sushi Cooler: sampled requests draft with MTP again (up to 28% more
throughput at two rows), four joined requests read their hyper-connections in one pass (11% faster), and prefill no
longer waits between chunks. DeepSeek-V4.1 prefills 8,192 tokens a chunk by default. - A thinking model never ends on an empty turn: when a model stops inside its thought with no answer or tool call,
sushi closes the thought and lets it continue once, on chat completions,/v1/messagesand Responses, and says so in
finish_details. On by default; turn it off in Settings › Replies or with--no-continue-after-thought. A client can
also continue a reply that holds only its thought. - Exact memory bills: every request reserves its cache at its last row before anything writes, the prefill's last
forward is billed at its full width, and a DeepSeek-V4.1 prompt of a million tokens is admitted and runs. Cached
conversations restored on the MLX engine no longer copy their cache every token. - Stricter, clearer APIs: logprobs list exactly the text a reply returns; a field of the wrong type is refused by
name; a reply cut inside a tool call never returns half a call; HTTP heads, chunked bodies and WebSocket frames follow
the protocol to the byte; every request after a shutdown gets the shutdown error. - Model management you can trust: one folder is one model however its path is spelled,
--max-resident-modelsholds
while two models load, a failed load names its own failure, andsushi kldwithout--ssd-budget-gbtakes serve's
memory default (fully resident on a 192 or 256 GB Mac). - Thanks to @balr0g and @NikolaKrstic94 for the logs that found both memory causes, and to @yanun0323 for the
thinking-turn report and the Strata numbers behind it.