oMLX 0.7.2

oMLX 0.7.2 brings faster DeepSeek V4.1 on every oQ build, Lightning MTP across Macs in distributed inference, and Qwen3.8-Flash-Next GGUFs on Splash. It's only been a day since 0.7.1, but these were worth shipping now.

Download: macOS 26 / 27 | macOS 15 Sequoia

Faster DeepSeek V4.1 on oQ2e to oQ4e

The fused decode kernels in 0.7.1 only read the MXFP weights of oQ4e, so oQ2e, oQ2.5e and oQ3e still decoded on a slower generic path. They now run on the fused kernels too, Lightning MTP verify shares work across the checked tokens, and prefill runs in 8192-token chunks. #4722

DeepSeek-V4.1-Flash-1017 -mtp checkpoints, 16K prompt, M3 Ultra 512GB.

With Lightning MTP + CED, prefill holds around 900 tok/s all the way to a 128K prompt.

Lightning MTP in Distributed Inference

Distributed inference now runs Lightning MTP on DeepSeek V4.1 and GLM-5.3 with pipeline or tensor splits. It turns on by the same rule as on one Mac, when the checkpoint has a draft head and the model setting does not turn it off. #4732

These runs used a TCP ring over Thunderbolt, not RDMA, so the absolute numbers are not the best case. Read them for the gain from Lightning MTP.

Decode with Lightning MTP off -> on, 512 generated tokens, measured before the #4722 kernels. Sampled is temperature 1.0, top_p 0.95, top_k 20.

• No more pipeline stall: an idle pipeline stage let the GPU slow down, which held DeepSeek V4.1 at about 1 tok/s across two Macs. It now decodes at about 28 tok/s with Lightning MTP off.
• Thunderbolt warnings: the plan preview warns when a Thunderbolt Bridge has only a self-assigned 169.254 address, which made the cluster fall back to the LAN or Wi-Fi without a word, and when the link runs over Wi-Fi.

More Models on Splash

With Splash 1.4.0, Qwen3.8-Flash-Next GGUFs run on Splash, and oMLX offers the matching incoai/Qwen3.8-Flash-Next-DFlash2 draft when it is missing. Splash 1.4.0 streams experts from SSD when memory is tight, and the Q2_0 build ran on a 64 GB M5 Pro. By @arnavprabhu in #4719, with a follow-up in 4f605d6.

• Split GGUFs: multi-part GGUFs at the repo root or in variant folders like Q2_0/ download, list and delete as one model, together with the shared vision projector.
• OptiQ checkpoints: mlx-community OptiQ builds that keep their vision weights in a subfolder are staged the way Splash reads them, so they start instead of failing with a 409 (#4718).

Other Changes

• Claude Code: notes that Claude Code sends after a tool result (hook context, loaded rules, reminders) stay in place instead of moving into the leading system block, so a turn no longer prefills the whole conversation again (@JarJarFran in #4712). Splash keeps later system messages in place too (@phaedrus1992 in #4708) and receives preserve_thinking (@isaac-cf-wong in #4594).
• Claude Code auto mode: classifier requests get a 256-token thinking budget, so reasoning models answer before the classifier deadline (f0e3f6a, with @phaedrus1992 from #4714).
• DFlash: ParoQuant Qwen3, Qwen3-MoE and Qwen3.5 targets can use DFlash (87251ad, with @popfido from #4685).
• SpecPrefill: a SpecPrefill request no longer shifts the RoPE positions of requests already decoding beside it, and sliding-window models no longer process the last prompt token twice (@billythegod in #4688). Gemma 3 global attention layers get the right rotation in sparse prefill (78d93f3).
• API: omlx launch works with an IPv6 bind host, and zero or negative values for max tokens, rerank top_n, embedding dimensions and TTS sampling fields are rejected (@billythegod in #4689, bed3f35).
• Attachments: browser files sent as application/octet-stream are converted by their extension (@saichowdary007 in #4179).
• Benchmarks: throughput tg rows decode the full length instead of stopping at the model's end of turn (@zviratko in #4687).
• Cluster: "Forget this Mac" no longer fails with a 500 (@bercedev in #4716).
• Web UI: the navbar menus and hover-only copy and unload buttons can be reached from the keyboard (#4729), console text meets WCAG AA and links use the grey scale (#4727), the dark theme shows the memory meter, plain commands and off switches (#4728), Status lines up with the other tabs (#4731), status card text stays inside narrow cards (#4706), and the log line count and hot cache unit read correctly (#4707), all by @LXD-8. The SpecPrefill link in model settings works again (@shaanchander in #4723), and Chinese bench metric cards lead with the abbreviation (@LXD-8 in #4699).
• Homebrew: formula updates no longer race when releases ship back to back (@billythegod in #4691).

Upgrade Notes

• Source installs: run make install (or make kernels) after updating. The sparse attention kernel changed in #4722.
• DeepSeek V4.1: prefill now runs in 8192-token chunks, so peak memory at a 16K prompt is 4-7 GB higher. With the SSD cache on, cache blocks are 8192 tokens, and a request that changes something inside the last 8192 tokens recomputes from that boundary.
• Cluster: Lightning MTP now turns on for deployments whose checkpoint has a draft head. Turn it off in Model Settings to keep the old behavior.
• Benchmarks: tg rows now decode the full length, so gen tok/s is not directly comparable with earlier runs of models that used to stop early.

New Contributors

@phaedrus1992 in #4708, @JarJarFran in #4712, @bercedev in #4716, @shaanchander in #4723.

Full changelog