What's Changed
- Optimize Qwen4 n-gram shard gathers by @pyros-projects in #2034
- Fix native quantized KV in Qwen MTP verification by @YuhuaBillChen in #1986
- optize_qwen4exp by @JJJYmmm in #2037
- Support Apodex 1.1 checkpoints by @Lazarus-931 in #2019
- Add MTP support for Qwen3.8-Flash-Next by @lucasnewman in #2040
- Anthropic route: keep in-conversation system messages in place (fixes prefix-cache defeat) by @Lazarus-931 in #2052
- fix(server): transcribe m4a and other non-encodable audio uploads by @Lazarus-931 in #2051
- fix(rfdetr): honor --show-boxes instead of ignoring it by @Anai-Guo in #2024
- fix(sam3): let --threshold fall through to the documented per-task defaults by @Anai-Guo in #2025
- Fix positioned top-k sampling for speculative generation by @tjansn in #2029
- Add continuous batching support for Qwen3.8-Flash-Next by @lucasnewman in #2056
- fix(qwen3.8): support official FP8 conversion by @Thump604 in #2058
- Add APC disk caching for Qwen3.8-Flash-Next by @lucasnewman in #2059
- Fix streamed responses after tool calls by @lucasnewman in #2062
- Fix tool call parsing with quotes and nested values by @lucasnewman in #2064
- fix(attention): materialize a quantized KV state before hand-rolled attention (#2060) by @Anai-Guo in #2065
- Fix nemotron_h prefill crash when inputs and inputs_embeds are both passed by @Lazarus-931 in #2057
- Add pr contributor reminder by @Lazarus-931 in #2068
- Redesign APC for dense and hybrid model caches by @Blaizzy in #1960
- moe-offloading by @Lazarus-931 in #1813
- fix(cache): trim ChunkedKVCache on valid length, not buffer width (#2031) by @Anai-Guo in #2036
- fix(recurrent_gemma): initialize nn.Module in LanguageModel by @Anai-Guo in #2075
- Bump version to 0.7.0rc0 by @Blaizzy in #2083
- perf(qwen3.8): adapt MTP draft depth by @Thump604 in #2046
- feat(qwen3.8): add external PLE storage by @Thump604 in #2045
- handling issue #2073, cache search locally by @Lazarus-931 in #2076
- lfm2_vl: load flattened OptiQ checkpoints (#2088) by @Lazarus-931 in #2092
- fix(video): resolve frame sampling in one place (#2050) by @Lazarus-931 in #2100
- Support float-quantized compressed-tensors checkpoints by @Lazarus-931 in #2102
- diffusion_gemma: strip channel scaffolding from decoded output by @Lazarus-931 in #2101
- fix: require compatible mlx-audio for VoiceChat by @Thump604 in #2110
- fix: stop generation on the tokenizer's EOS when the config omits it by @Lazarus-931 in #2112
- Add LLaVA-OneVision by @Lazarus-931 in #2115
- Fix phantom KV in ragged batched speculative rollback by @Lazarus-931 in #2113
- Add DeepSeek-V4 DSpark speculative drafter by @Lazarus-931 in #2097
- Add LongCat-Flash-Lite-Sparse (LSA + n-gram) by @Lazarus-931 in #2063
- muse_glimmer: load flattened OptiQ checkpoints (#2108) by @Lazarus-931 in #2118
- fix(speculative): detect native MTP drafters via num_nextn_predict_layers by @Lazarus-931 in #2120
- fix(generate): chunk prefill for prompts shorter than prefill_step_size by @Lazarus-931 in #2119
- GLM-5.3 - 2026-08-28 by @Lazarus-931 in #2078
- GLM-5.3 MTP - 2026-08-28 by @Lazarus-931 in #2079
- Fix TurboQuant batch decode, MTP KV quantization, and attention sinks by @YuhuaBillChen in #1822
- Add OmniParser v2 YOLO11 support by @Lazarus-931 in #2122
- fix(florence2): load legacy large-ft checkpoints on transformers>=5.14 by @Lazarus-931 in #2123
- Add Video Depth Anything (video_depth_anything) by @eklipse2k8 in #2104
- Strip Muse Glimmer channel scaffolding from generated output by @Lazarus-931 in #2129
- sam3: fix pred_boxes inflation and offset (RoPE scale, RPB grid, ref_point_head) by @Lazarus-931 in #2128
- Add OpenAI Privacy Filter model support by @lucasnewman in #2131
- Support multiple tool call invocations at once by @lucasnewman in #2132
- Require MLX 0.32.2 for faster speculative decoding by @Blaizzy in #2133
- Add DSpark support for Gemma 4 by @Lazarus-931 in #2137
- qwen3_vl: render per-group video timestamps like the reference processor by @Micha0827 in #2142
- Add DeepSeek-V4 Flash Vision EXP support by @byoan in #2136
- granite4_vision: align deepstack injection with chunked prefill via c… by @Lazarus-931 in #2143
- Register the qwen3_vl_embedding auto-processor patch by @Lazarus-931 in #2146
- Optimize exact Qwen verifier QMV unrolling by @Blaizzy in #2140
- Keep quantized APC checkpoints packed by @Blaizzy in #2090
- Normalize tokenizer padding in BatchGenerator by @Blaizzy in #1951
- Add Nemotron-H speculative decoding support by @Lazarus-931 in #2145
- Enable speculative decoding for Laguna by @Lazarus-931 in #2151
- Fix ArraysCache trim crash on multi-turn reuse for hybrid models by @Lazarus-931 in #2152
- mla: use the absorbed path for multi-token steps by @stooit in #2150
- Fix Gemma4DSparkAttention AttributeError on first forward by @Lazarus-931 in #2158
- Porting model K2-Horizon by @Lazarus-931 in #2153
- Add Ornith 1.5 (Qwen3.5 VL) support and verify qwen3_5_moe MTP by @Lazarus-931 in #2125
- Optimize Qwen4 QSA cache reuse and batched parity by @Blaizzy in #2126
- Fix GLM and Kimi output control tokens by @Lazarus-931 in #2170
- Add MoGe-3 monocular geometry estimation model by @eklipse2k8 in #2171
- Add Z1T-0 (Extropic attention-free sparse LM) by @Lazarus-931 in #2166
- Fix Unlimited-OCR prompt suffix by @Lazarus-931 in #2169
- fix(aya_vision): reference patch_embeddings in the spatial_shapes branch by @Anai-Guo in #2173
- Freeze base model when resuming LoRA training via --adapter-path by @Lazarus-931 in #2181
- fix: declare jinja2 as a runtime dependency by @Shouta0108 in #2176
- Bump version to 0.7.0 by @lucasnewman in #2175
New Contributors
- @pyros-projects made their first contribution in #2034
- @YuhuaBillChen made their first contribution in #1986
- @tjansn made their first contribution in #2029
- @eklipse2k8 made their first contribution in #2104
- @Micha0827 made their first contribution in #2142
- @stooit made their first contribution in #2150
- @Shouta0108 made their first contribution in #2176
Full Changelog: v0.6.17...v0.7.0