oMLX 0.7.1.dev2

This is a development release. Please report any bugs through GitHub Issues. If you need a stable release, please use oMLX 0.7.0.

oMLX 0.7.1.dev2 helps first-time users find a model that fits their Mac without guesswork, serves Qwen3.8 and Qwen3.6 through Splash, reworks model settings and chat in the web UI and the macOS app, and runs DFlash inside the batched engines.

Download: macOS 26 / 27 | macOS 15 Sequoia

Find the Right Model for Your Mac

No more guessing which model to download first. The new Recommended tab in Models > Downloader checks what your Mac can actually run and shows only oQ models that will load, from the best fit to bigger models with the setting that unlocks each one. Models with Lightning MTP have it on by default, so they are fast right after download. #4468

Splash as an External Engine

Qwen3.8 and Qwen3.6 models can now be served by Splash, a Metal engine with DFlash2 speculative decoding, running in its own process. The model stays in oMLX's model list, API, dashboard, cache stats and benchmarks. Turn on "Use external engine" in Model Settings > Advanced: oMLX finds the Splash 1.3.0+ launcher, hands it the model and DFlash2 draft from your model directories, and fits its memory and context window to the oMLX limit. Vision checkpoints take image input too (@Flo5k5 in #4537). #4447, built on #4133 by @paperniuk, with M4 Max runs by @e-Garcia in #4154.

Reworked Model Settings, Chat and Logs

Model Settings in the web UI is grouped into sections with a side menu, and every setting is one row with a short description (#4415). The macOS app uses the same grouping as System Settings-style pages that save as you edit, and adds the settings it was missing (#4530). The chat page moves the model, profile and reasoning level into the composer, with a reasoning slider that offers the levels each model accepts (#4446). The app's Logs screen reads as a table of records with a raw view and a resizable entry card (@LXD-8 in #4514, #4531).

Faster GLM-5.3, DFlash and TurboQuant

DFlash now runs inside the batched engines instead of its own single-stream engine, so DFlash requests get continuous batching and the hot and SSD prefix cache, and dflash-mlx is no longer a dependency. It covers Qwen3.5/3.6/3.8, Qwen3, Laguna, MiMo V2, GLM-5.3, Gemma 4 and Muse Glimmer (#4508). DSpark drafters now use their own block layout and Markov head, which lifts acceptance on Qwen3.8-27B from 16-21% to 61-85% (@fneto1977 in #4525).

Each figure is measured against the code right before the change. By @junmo-kim (#4523, 1019201, #4386), @alan13367 (#4408), @e-Evolution (#4442, and #4441 behind #4453), @AureliusClaw (#4376) and @williamxie1989 (147b8bf). Also faster: CausalLM rerank, 2x to 7x (@yanzhaohui1999 in #4407), Lightning MTP acceptance after a GLM-5.3 prefix cache hit is back to cold level (@feni6 in #4540), Qwen4 PLE reads and wide 8-bit verify projections (@Ambisphaeric in #4464, #4472, #4475), and MoE route inversion (@samfenwick, 125ca68).

New Features and Improvements

• Requests survive memory pressure: running requests are paused and resumed instead of aborted, and cancelled prompts stay cached. Resending a prompt cancelled 60 s into a 38K prefill cuts TTFT 106 -> 40 s (-62%) (#4509, from @e-Evolution's analysis in #4440).
• Decisions: POST /v1/decisions follows OpenAI's Decisions API on any chat or vision model (@PowerSpy in #4436), /v1/systemone also answers with any chat model (#4552), and the Decision 3.0 models (d3, d3-flash, d3-mini, d3-nano, d3-lite, d3-edge) read images and videos (#4470).
• Long context: Qwen3.5-family models use static YaRN when the context window goes past the native 262K (#4445, with @VictorLowther).
• oQ: streamed oQe calibration for GLM-5.3-Flash on 128 GB Macs (@cropduster in #4174), DeepSeek V4.1 oQ2 to oQ2.7 with 4-bit lookup tables and bit tables in uploaded model cards (#4448), EmbeddingGemma 2 (aeb88a3), and Gemma 4 12B assistant MTP heads (166e9fd).
• Sampling and usage: no_repeat_ngram_size and ngram_window now apply (@Cassian433 in #4510), and chat usage reports reasoning_tokens (@yanzhaohui1999 in #4546).
• Network access: an "Allow inference without API key" toggle for network binds, behind a warning. "Skip API key verification" is now "Disable all authentication" (#4528).
• Integrations: omlx launch vibe for Mistral Vibe 2.26.0+ (@samfenwick in #4367), and context overflows return a standard context_length_exceeded error that clients use to compact (df7fa9e).
• Models: OpenAI content parts in /v1/embeddings (@zviratko in #4419), MiniCPM-V 4.7 with MoE expert offload (@Woody-Hu in #4549), and Qwen4 Extended Fused Decode in Model Settings (3c39990).
• Web UI and app: Downloaded badges in the model browser (@jandrop in #4431), copy buttons on every model row (@mudravadev in #4521), deleting models from the Hugging Face cache (#4548, following @popfido's #3029), MTPLX side-car import in the app (@popfido in #2815), login focus and inset select arrows (@mudravadev in #4497, #4498), PP/TG/TTFT in capitals, Chinese 万/亿 counts and the settings menu edge (@LXD-8 in #4451, #4465, #3826, #4432), switch states for screen readers (@yanzhaohui1999 in #4557, c0239d4), and Czech translations (@zviratko in #4522).
• Other: MLX 0.32.4 with native kernels built through MLX's MetalExtension (@XXXXRT666 in #4550), mlx-vlm 0.7.8 (#4529), the Chinese README (@JasonYeYuhe in #3329), and the MCP restart note (@ketpatil77, e0c24bf).

Bug Fixes

• Lightning MTP no longer aborts a batch with an IndexError after its draft depth drops (@junmo-kim in #4410).
• Stop strings end the answer, not the reasoning, on chat and messages (@HamedShams in #4352), and thinking_budget closes thinking the way the template renders it on Qwen3.6 and Gemma 4 (@richgoodson in #4539).
• DeepSeek V4.1 tool calls that end without the outer close tag come back as tool_calls (91718c9), V4 reasoning no longer repeats in content on tool-call turns (@williamxie1989, 611540b), and Qwen XML tool calls keep anyOf string/null values as strings (@yanzhaohui1999 in #4544).
• Cache fixes: deduplicated prefix blocks are pinned before reuse (#4443 with @billythegod), prompts shorter than one block hit the cache (@Woody-Hu in #4428, c756df2), LFM2 reuses prefixes (@alan13367 in #4532), padded prefill keeps GDN state exact (#4526), and long-running Qwen3.5 batches no longer hit the Metal resource limit (41166af).
• TurboQuant batches no longer let left-padded rows read their padding in Qwen3.5/3.6 decode (#4453).
• Memory guard fixes: prompts that fit are no longer rejected right after a request frees memory, a rejection keeps the cache for the retry (#4554), queued requests wait while running ones make progress (a987407), and fp16 Qwen3.8-Flash-Next builds use less prefill memory and reach longer contexts again (89a40d1).
• /v1/systemone no longer returns 500 when the decision model is unloaded during encode (013bc3f), and the ANE tuner keeps the verified result when its last step fails (e292a3b).
• A failed STT load no longer hangs the server (@Flo5k5 in #4542). The macOS app no longer blocks logout, restart and shutdown (e860713), and it picks up an API key set after launch (8e1e386).
• VLM requests with back-to-back user messages work on strict templates (64bf5d1), and OptiQ VLM checkpoints load as VLMs again (@Woody-Hu in #4549).
• Downloads report gated repos as gated (e870cb9 with @yanzhaohui1999), and ModelScope rows show the right owner and size (@yanzhaohui1999 in #4555).
• Cluster: hybrid models without layer_types reserve KV only for attention layers, pipeline ranks stay up during long prefills (#4553), and CUDA workers match MLX 0.32.4 (c73e8fc).
• oQ jobs can no longer write to the same output folder (#4443), DeepSeek V4.1 oQ3 runs again (3f99250), and GLM-5.3 oQe calibrates its MTP head (7388337).
• LiveCodeBench grades functional problems by return value (@yanzhaohui1999 in #4545).

Upgrade Notes

• Source installs: run make install (or make dev) after updating. oMLX now uses MLX 0.32.4, and native kernels built for the previous MLX are rejected and fall back to slower paths (#4550). Upgrade every Mac in a cluster at the same time.
• DFlash: dflash-mlx is removed. Settings only the old engine read (DFlash in-memory and SSD caches, dflash_max_ctx, sink size, draft quantization) are gone, a saved ddtree verify mode loads as adaptive, and changing a DFlash setting reloads the model (#4508).
• Sampling defaults: models without saved sampling settings now use the values in their generation_config.json. Qwen models without top_k/top_p get 20 / 0.95 and Gemma models 64 / 0.95. Saved settings are not changed (#4416).
• YaRN: a per-model context window above 262,144 on a Qwen3.5-family model now turns on YaRN for every prompt, so keep the window near the length you need (#4445).
• LiveCodeBench: scores from earlier versions were graded the old way and are not comparable (#4545).

New Contributors

@jandrop in #4431, @mudravadev in #4497, @Woody-Hu in #4428, @Ambisphaeric in #4464, @Cassian433 in #4510, @feni6 in #4540, @Flo5k5 in #4542, @fneto1977 in #4525, @XXXXRT666 in #4550.

Full Changelog: v0.7.1.dev1...v0.7.1.dev2