This release includes extensive internal changes from upgrading core libraries. If you encounter issues with a previously working model, please open a GitHub issue with logs and switch back to dev2 for now. I plan to test this version for 1-2 days, then proceed with an RC followed by a stable release.

If your server won't start after upgrading, a bug may be affecting externally accessible servers with API key auth disabled. Please enable auth in settings.json or reset all settings. A future release will show a warning to help you resolve this.

oMLX 0.7.0.dev4

oMLX 0.7.0.dev4 brings one-click model settings from omlx.ai benchmarks, faster DeepSeek V4.1 prefill with CED, multi-request Lightning MTP, and a customizable dashboard. This release also updates core model libraries and adds layer-streaming oQe calibration and Spark-X2.5 support, alongside fixes for tool calling, vision caching, and distributed serving.

Download: macOS 26 / 27 | macOS 15 Sequoia

One-Click Model Settings from Benchmarks and Recipes

oMLX.ai hosts over 450,000 benchmarks submitted by real users. Now you can apply the fastest benchmarked settings for your Mac and model directly from oMLX, in both the web dashboard and macOS app.

• Apply optimal settings finds benchmarks matching your model and Mac chip. Choose a top result for prefill or generation speed and apply its settings with one click.
• Apply custom recipe works even when your customized model has no benchmark results. Find a model with the same architecture on oMLX.ai, copy its one-line recipe, paste it into oMLX, and click Apply to reuse the settings.

See #3701.

DeepSeek V4.1 CED Prefill - Up to 79% Faster Prefill

DeepSeek V4.1 now supports CED prefill, which reduces decoder-side attention and MoE work to a trailing token window while retaining full-context global KV from the encoder. This speeds up prompt processing, particularly for long inputs.

CED is off by default and available under per-model Experimental Features. It uses an approximate bounded-replay path and can change model outputs.

Benchmarked on an M3 Ultra with 512 GiB of unified memory, using DeepSeek-V4.1-Flash-oQ4e-mtp with Engram in RAM, Code (Python) context, temperature 1, and top_p 1. SSD KV and hot prefix caching were disabled.

Prefill and peak memory were measured with MTP ON. Generation columns compare MTP OFF and ON with CED enabled, using 128 generated tokens.

See #3607 and the benchmark comment.

Multi-Request Lightning MTP

Lightning MTP now supports concurrent requests on supported Qwen, DeepSeek V4.1, and GLM-5.3-Flash adapters, extending the single-stream support in dev2.

Unsupported architectures retain single-request MTP and use ordinary decoding for multiple requests.

Benchmarked with two concurrent requests on an M3 Ultra with 512 GiB of unified memory, using MLX 0.32.2 and oQ4e model bodies.

This builds on the fused-batch approach introduced by @gaineyllc. See #3695.

Customizable Dashboard Layout

The dashboard now supports custom layouts. Move, resize, hide, and restore status blocks, and adjust the dashboard width to fit your screen. Layouts are saved across sessions.

See #3694.

Other New Features and Improvements

• Updated mlx-lm and mlx-vlm pins. Adapted batching, Lightning MTP, and cache restoration to the updated libraries while preserving existing SSD caches. (#3708, #3719)
• Faster admission for new requests during generation. Prevent long chunked prefills from repeatedly delaying queued requests while decoding is active. (#3726)
• Global settings reset in the web dashboard and macOS app. Restore defaults for review before saving or applying, while keeping paths and API keys. (47de6f2b)
• Layer-streaming oQe calibration for models larger than RAM. Calibrate supported MiniMax-M3 and Qwen3.8-Flash-Next checkpoints one layer at a time. (#2161)
• Spark-X2.5 model support. Includes reasoning output and structured tool calls. (#3606)
• Improved MoE expert SSD offload. Read missing experts in parallel and reduce repeated prefill fetches, with safer read-failure recovery. (#3589, #3654, #3628)
• Decoded-image caching for repeated screenshots and photos. Avoid repeated image decoding in multi-turn conversations. (#3634)
• Better accuracy benchmark scheduling. Worker slots refill immediately, and code scoring no longer blocks the event loop. (#3635)
• Enhanced Readability in the macOS app. Added an Appearance toggle for higher-contrast text and indicators. (#2743)
• Improved dashboard accessibility. Enabled mobile zoom and improved screen-reader labels and keyboard focus. (#2669)
• Updated macOS status layout. Active Now appears above Usage History. (#3584)
• Expanded interface translations. Added remaining UI strings to the catalog and updated Simplified Chinese translations. (#3682, #3684)

Bug Fixes

• Require API authentication for network access. LAN and 0.0.0.0 binds now require an API key. (09a7c437)
• Fixed batched Lightning MTP corruption at cache boundaries. Keep each request's boundary update isolated from other rows. (#3724)
• Fixed GLM pooled-cache memory growth and restoration. Avoid cumulative KV snapshots and preserve complete pooled history across cache reuse. (#3290, #3710)
• Fixed Qwen4 optimization recovery. Retry optimized execution after transient failures instead of leaving it disabled. (#3723)
• Fixed Lightning MTP request joins and cache handoffs. Avoid unnecessary history rebuilds, recover failed handoffs before resuming, and exclude prefill from batch cost measurements. (#3702)
• Fixed handling of unrecoverable Metal errors. Exit when Metal rejects further GPU submissions instead of continuing in a broken state. (#3706)
• Fixed distributed worker failure recovery. Failed workers no longer leave reloads and unloads stuck behind stale busy state. (#3631, #3663)
• Fixed Responses namespace tools. Preserve grouped tools, namespace information, and tool-call history across turns. (#3664, #3371)
• Fixed Qwen tool-call recovery across APIs. Preserve recoverable and unknown-tool calls; report malformed calls instead of empty successful responses. (#844, #3693)
• Fixed XML tool-call parsing. Support MiniCPM5-style calls, CDATA literals, dotted or hyphenated names, and fragmented array arguments. (#3530, #3383, #3624)
• Fixed Responses and Anthropic compatibility. Honor disabled tools, retain MCP tools, and preserve plain-text documents. (#3383, #3617)
• Fixed text-only VLM checkpoint loading. Load checkpoints without vision weights without creating a nonexistent vision tower. (#3698)
• Fixed Bonsai weight-format detection. Ordinary 2-bit affine checkpoints no longer receive incompatible packed-weight loading patches. (#3716)
• Fixed vision prefix-cache boundaries and mixed-image cache reuse. Prevent invalid token boundaries and incompatible cached-feature concatenation. (#3675, #3512)
• Fixed recurrent cache batching and extraction. Handle Qwen linear-cache conversion and empty VLM cache slots safely. (#3703, a3524a9c)
• Bounded oversized image processing. Validate upload sizes and downscale large images to limit memory use. (#3661, #3650)
• Fixed Qwen ANE prefill synchronization. Prevent subsequent tiles from overwriting pending hybrid merge inputs. (#3588)
• Fixed Lightning MTP sampling and checkpoint compatibility. Respect XTC sampling, reject invalid processor contracts, and honor MTP-specific expert counts. (#3084, #3511, #3659)
• Improved chat responsiveness during embedding and reranking workloads. Added decode fairness and fixed ModernBERT null relevance scores. (#2873, #3633)
• Fixed profile validation and app/web consistency. Validate saved settings, apply current templates, and repair stale references. (#3674, #3678)
• Fixed PDF processing when the selected OCR model is missing. Fall back to MarkItDown and show the unavailable model in settings. (#3709)
• Fixed macOS memory statistics compatibility. Use portable system calls while preserving compressed-memory reporting. (#3358, #2983)
• Fixed model resource cleanup. Allow progressing cache writes to finish and make disk-backed Qwen PLE resources fork-safe. (#3673, #3627)
• Improved cache diagnostics. Distinguish expected cache-store skips from failures and report failed file cleanup. (#3699, #2668)
• Fixed DeepSeek V4.1 checkpoint discovery and export metadata. Recognize Hugging Face cache checkpoints and mark exported shards as MLX format. (#3629, #3583)
• Fixed benchmark scoring and exports. Honor explicit multiple-choice answers and preserve literal text in exports. (#3668, #3682)
• Improved cluster discovery. Skip inactive network interfaces and log failed peer handshakes. (b702f6f2)
• Fixed macOS server port persistence. Port changes survive app restarts. (#1215)
• Fixed minor UI inconsistencies. Match login and chat icons to the navbar and apply dark-theme usage table borders. (3acc260e, 3a101ea8)
• Fixed Claude plan-mode routing. Disable automatic classification during plan mode. (fb9df390)
• Fixed canceling global settings resets. Restore the previous values when a reset is canceled. (cbc1a803)

Contributors

@williamxie1989, @ernestas-poskus, @beatakouchnir, @Rascal, @L4XB, @mensaprodigy, @nisargnegi, @xunlinkx, @pierre427, @ccdsky, @osviel91, @monroewilliams, @djtech42, @beaglemoo, @joergbirkhold, @LittleYier, @astutegrogan, @mrtkrcm, @jonathan308, @alytaphoenix, @LXD-8, @MaxFreedomPollard, @X-Wei, @builtbycnob, @saichowdary007, @tc3oliver, @luken.

New Contributors

@ernestas-poskus, @mensaprodigy, @nisargnegi, @pierre427, @ccdsky, @osviel91, @djtech42, @joergbirkhold, @LittleYier, @astutegrogan, @X-Wei, @tc3oliver, @luken.

Full Changelog: v0.7.0.dev2...v0.7.0.dev4