github incoai/splash 1.7.1
Splash 1.7.1

37 minutes ago

Splash 1.7.1 brings faster Neural Engine prefill, better cache reuse across conversations, and up to eight concurrent requests.

Engine

  • Faster Neural Engine prefill. Qwen3.8-27B has about 23% lower TTFT on M6 for cold 2K and 8K prompts, and 6–13% lower TTFT on a 16-core M5 Pro for tested prompts of 1K tokens and above. The ANE and GPU now overlap more of their work.
  • Faster TTFT across conversations. New chats sharing a system prompt and tools have 2.5–2.9× faster TTFT on tested 27B, 35B and Flint configurations. Branches and regenerated replies also reuse tokenized messages.
  • Up to eight concurrent requests. Set --max-concurrent-requests or use Concurrency limit in the app. With eight clients and 2K prompts, Qwen3.6-35B-A3B delivers 13% higher throughput than four lanes on a 64 GB M5 Pro. The default remains four.
  • Faster weight loading and compute. Cached weights load up to 1.54× faster on M5 Max; Flint GGUF loading drops from 11.1 to 3.3 seconds on M5 Pro. MoE and streamed-expert prefill are faster, and Bonsai's four-request decode step is 1.34× faster on M6.
  • More reliable serving. Fix streamed JSON schema failures and Neural Engine calibration keeping a false “no gain” result. Busy calibrations are rechecked, and the app now offers Recalibrate Neural Engine. The automatic Metal budget leaves 10% of system memory unwired.

Models, server and Mac app

  • Load Flint and Qwen3.8-Flash-Next MLX checkpoints, including Flash-Next vision. Text-only MLX models install without --language-only.
  • Set server-wide sampling defaults with --sampling-default NAME=VALUE. API usage explains cache reuse, and /status and /metrics report SSD reads.
  • Unsent drafts and attachments survive app restarts. Downloads account for cached files, check available disk space, and show clearer errors.
  • Chat rendering improves tables, bold text beside CJK characters, formulas and links.

Upgrade

Choose Splash → Check for Updates… or download the DMG. With Homebrew, run brew update, then brew upgrade splash for the CLI or brew upgrade --cask splash-app for the app.

The first model start recalibrates the Neural Engine. Higher concurrency can reduce available context on Macs with limited memory. More MLX MoE and Bonsai operations use INT8 activations on M5 and later, so outputs can differ. Use --activations bf16 for BF16 activations and --disable-ane to run entirely on the GPU.

Thanks to @liang2kl for contributions to Neural Engine acceleration, to @pcvelz and @dmkjr for their contributions, and to @ardnt, @tomchk, @brainnoms, @alron, @ghisguth and @janhilgard for their reports and ideas.

Detailed changes and benchmarks · Full changelog

Don't miss a new splash release

NewReleases is sending notifications on new releases.