oMLX 0.7.3

oMLX 0.7.3 makes DeepSeek V4.1 prefill and batched decode faster, lets Ask AI hold voice conversations, search the web, read links and look at your screen, and cuts memory use when many requests run at long context. The notes below cover changes since 0.7.3rc1, and earlier 0.7.3 highlights are summarized at the end.

Download: macOS 26 / 27 | macOS 15 Sequoia

Ask AI Voice Mode, Web Search and Screen Capture

Talk to your local models. Turn on Voice under App > Ask AI, press Option-Shift-Space and speak, and the answer plays back with captions. Ask AI can also search the web, read links and look at your screen (#4976).

Faster DeepSeek V4.1

DeepSeek V4.1 Flash prefills much faster on M5 with INT8 Activation Prefill in Model Settings (#4946) and Engram prefetch (@N1k1tung in #4993). Batches of more than 8 requests now stay on the fused decode kernels (@junmo-kim in #4989).

INT8 rows: oQ2e-mtp with the setting off and on (off, on, on).

Lower Memory and Faster Decode with Many Requests

Many long conversations at once now use far less memory, because the batch KV cache is updated in place instead of copied (#4916, #4936, #4954, #4971). Batched decode on oQ and affine models reads each weight once for all rows (#4977, #4990), and hybrid Qwen models reuse a shared system prompt across conversations (#4994).

Update Downloaded Models

Check for updates in the model manager, then fetch only the files that changed on Hugging Face instead of downloading the whole model again (#4949).

New Features and Improvements

• DeepSeek V4.1: Macs before M5 get the faster HC projection and decoder skip, +3-4% prefill on an M3 Ultra (#4946). Expert offload reads missed experts straight into their slots (@cropduster in #4951), and small-row HC projections stay FP32 on the fused kernel (fac5de3 with @williamxie1989).
• Ask AI: prompt placeholders such as {date}, {app} and {clipboard}, Enter to send (@tsang in #4962), and a /v1/realtime endpoint that follows a subset of the OpenAI Realtime API (#4976).
• oQ: full-attention layers of hybrid dense models get one more bit, and the oQe calibration now follows each model's own forward. KL drops 9-49% for 2.8-4.5% larger text weights (#4930). DeepSeek V4.1 oQ3e keeps attention and indexer tensors at 8-bit, with 7.8% lower KL for +1.3 GB (6d6a417), and Gemma 4 E2B and E4B builds apply their imatrix (@vichzp in #4625).
• Models: LFM3-1.7B, LFM3-Audio-2B and the LiquidAI d1-2B and d1-Omni-2B decision models (#4937 with @Arpit1980gh). LFM3-Audio and LFM3-Omni serve /v1/audio/transcriptions, /v1/audio/speech and /v1/audio/process, Qwen3-Omni loads on the VLM engine (#4976), and Unlimited-OCR-2 comes with mlx-vlm 0.7.11 (#4934).
• Bonsai: 1-bit and 2-bit layers run on stock MLX kernels. Bonsai-27B 1-bit prefill on a 128-token prompt goes 100 -> 362 tok/s (+262%), and decode goes 66.1 -> 70.5 tok/s (+7%) (#4972).
• M1 and M2 Macs: 6-bit and 8-bit layers use the verify MMA kernel (@fneto1977 in #4898).
• API: ignore_eos on /v1/chat/completions and /v1/completions (@starpit in #4978), also applied to the stop tokens of output parsers (96361a5).
• Web chat: temporary chats that never reach history (@LXD-8 in #4910).
• GGUF: faster header reads (efa950a with @LXD-8).

Bug Fixes

• The prefill memory guard no longer gets stuck after overlapping requests and rejects every normal prompt until restart (#4903).
• SSD cache: when the configured dir is unavailable, the SSD cache turns off until restart and the web dashboard and the macOS app show a warning, instead of writing to the internal disk (e6d5989). DeepSeek V4.1 CED prefill shares the same SSD budget (#4912), and the dashboard counts old-layout blocks in cache size (1855ba7 with @FazeelUsmani, #4911).
• Lightning MTP: verify on M1 and M2 5-bit models no longer fails with Thread group size (512) > max 448 (#4913). Imported OptiQ and zero-centered MTPLX side-cars get their head norms shifted, so Qwen3.5-9B draft acceptance goes from 0% to 63-82% (4848f57 with @begcdn, #4960). Qwen3.8-Flash-Next oQ builds with an 8-bit out_proj take the fused decode path again with MTP off (#4988).
• Tool calls: bare <invoke name="..."> calls are parsed, and MiMo uses the qwen3_coder parser (#4933). A max_tokens cut inside a Qwen tool call finishes with length instead of incomplete_tool_call (b2f3239).
• Cluster: CUDA ranks no longer deadlock on the first decode step (@desmccrory in #4969, #4743), the Engram SSD offload setting reaches pipeline ranks and counts as SSD-resident (@digitalfactory-ako in #4985, #4984), and a failed launch no longer breaks the next one (@digitalfactory-ako in #4974, #4973).
• Downloads: in repos that publish several precisions side by side, only the default checkpoint is required, so the other precision folders can be unchecked (@Woody-Hu in #4967).
• oQ: the source model list skips entries the server cannot read, such as .Spotlight-V100 on external drives (@jasonpaulso in #4961).
• A failed manual model load returns a matching status code and reason instead of a 500 (@LXD-8 in #4948).
• Web UI: saved chats are kept when a chat starts before history loads (@LXD-8 in #4915), and the disable-all-authentication switch is tinted red (@LXD-8 in #4899).
• macOS app: the Logs header inset is back (@LXD-8 in #4900), bench text exports line up their columns (@LXD-8 in #4939), and the accuracy Running row clears after Cancel (@burghr in #4987, #4250).

Highlights from 0.7.3rc1 and the Development Releases

• Ask AI. A question bar for your local models in any app, with slash presets and image input (#4854).
• GPU Power Limit. Caps the GPU's power while a model generates, for working on battery (#4855).
• Faster Qwen3.8-Flash-Next and Ling 3.1 Flash. Ling 3.1 Flash with Lightning MTP up to +82% (#4881), and Flash-Next decode +24% (#4880).
• MiniMax M3.1 Flash. With its DSpark draft as Lightning MTP, up to +87% for one request (#4777).
• Faster M1 and M5 Macs, better low-bit oQ. About 50% faster long-context prefill on M1 Ultra (#4496), DeepSeek V4 prefill attention on the M5 tensor units (#4808), and up to 73% lower KL at the same size (#4776).

Benchmarks and every contributor credit are in the 0.7.3rc1, 0.7.3.dev2 and 0.7.3.dev1 release notes.

Upgrade Notes

• Source installs: run make install after updating. mlx-vlm moves to 0.7.11 and mlx-audio to 0.6.3 (#4934), mlx-lm to 0d4d3d04 and modelscope to 1.28 or newer, and transformers up to 5.20 is allowed (#4834).
• INT8 activation prefill: the settings qwen35_oq_a8_enabled and qwen35_oq_a8_min_tokens are now a8_prefill_enabled and a8_prefill_min_tokens. Saved settings, profiles and recipes with the old names are migrated on load (#4946).
• oQ: new quants at the same level get a different bit split, and their text weights can be 2.8-4.5% larger (#4776, #4930). Existing quants load and run as before.
• SSD cache: blocks that earlier versions saved are not reused for gpt-oss, ERNIE 4.5 MoE, PhiMoE, PLaMo 3, FP8 Ministral 3 and Muse Spark (#4778), Ling Flash (#4881), and DeepSeek-OCR encoders, MiniCPM-V 4.6/4.7 vision and per-expert NVFP4 MoE (#4934). DeepSeek V4.1 CED blocks move from the deepseek_v41_ced_v1 subdirectory into the shared cache dir, and the old subdirectory is removed at startup (#4912). If the SSD cache dir is unavailable, the SSD cache is off until restart (e6d5989).
• DeepSeek V4 and V4.1: the M5 tensor-unit prefill kernel is not bit-identical to the old one (#4808), and batches of more than 8 V4.1 requests now decode through the fused kernels (#4989), so greedy output can differ at near-ties.
• Cache stats: prefix hit rates can read lower than before, since an evicted or partial restore no longer counts as a full hit (#4782).
• Benchmarks: CMMLU, JMMLU and KMMLU scores can be higher than in earlier runs (#4774).
• API: an empty completion prompt now returns 400, and admin output token defaults must be 1 or more (19178d0). logprobs on an engine that cannot produce them (cluster, Splash, diffusion) returns 400 (#4859). An unclosed tool-call envelope returns incomplete_tool_call instead of stop (#4869), while a max_tokens cut returns length (b2f3239). /v1/messages/count_tokens can read higher where tools or thinking settings apply (#4875). Qwen3-TTS without max_tokens takes its limit from generation_config.json (#4934).
• macOS app: updates now download in the background by default and still ask before installing. Turn off Download Automatically under Updates on the Status screen to only get notified.

New Contributors

@jwiegley in #4797, @kathsucurry in #4795, @wesbiggs in #4769, @Abelo9996 in #4753, @ChristianPraiss in #4737, @sampe-bot in #4804, @Phenom300 in #4808, @BrownMark in #4815, @parasew in #4819, @HeinrichvH in #4869, @tsang in #4962, @desmccrory in #4969, @digitalfactory-ako in #4985, @burghr in #4987, @starpit in #4978.

Changes since 0.7.3rc1 | Full changelog from 0.7.2