beamivalice/sushi v1.6.0

sushi v1.6.0

on GitHub

v1.6.0 — Stability, UI and Architecture Refactor

• Qwen3.8-Flash-Next prefill keeps its speed on 64 GB Macs: when macOS pushes the n-gram table out of memory beside
  the weights, prefill now keeps its parallel reads instead of walking the table from the SSD, which had halved its
  speed since 1.5.1; QWEN4_PLE_PREFETCH_PREFILL=1 is no longer needed. Thanks @wjjchen for the report.
• Context extension is a per-model setting: the Models page runs Qwen3.8-Flash-Next with YaRN ×2 or ×4 over its
  native window (yarn_factor in model-settings.json), shows the resulting window before loading, and keeps separate
  prompt-cache entries per setting, so a 256 GB Mac can serve the BF16 pack or a custom pack at long context. A
  --ctx-size above the window is refused. Thanks @Cianidos for the request.
• Nothing is silently clamped or ignored: a request value out of range (temperature, top_p, top_k, penalties,
  logprobs, seed, a negative max_tokens) or a strict json_schema the server cannot enforce is a 400 that says
  why, and a launch flag that cannot work as given (--mtp with --no-mtp, an empty --api-key,
  --max-concurrent 0) exits with an error instead of being overridden.
• A failure no longer outlives its cause: a model refused for its SSD budget or memory limit loads again once the
  setting is lowered, with no restart; Load retries a failed model with the current settings; a refused Save clears when
  the value is fixed; and an SSD prompt cache declined at load (a full volume) re-attaches by itself. A model's page
  shows the Mac's three wired-memory limits with one copyable sudo sysctl command (thanks @Cianidos).
• A sturdier web page: settings are checked before they are sent, errors name their remedy instead of showing
  Offline, the chat opens on the model loaded last and follows the loaded model, a message can be edited and resent,
  each pack shows its in-memory and full weight size, the models folder is picked on the Models page, and the markdown
  renderer draws task lists, strikethrough, table alignment and bare links.
• Fewer first-chat failures: the first chat after loading a Flash-Next pack that streams its experts, an unloaded
  DeepSeek-V4.1-Flash missing from the chat page, a burst of reloads that loaded one model twice, and an MLX error left
  over from a load no longer fail a request; a request that fails inside the engine now logs why.
• For hosts that vendor the engine: each architecture is a self-contained module (-Darchs builds without any of
  them), sushi_exl3 names no architecture, its deprecated aliases are gone, and its re-vendor contract is written down
  with a version constant. Model output is unchanged.