What's Changed
- Optimize Qwen4 n-gram shard gathers by @pyros-projects in #2034
- Fix native quantized KV in Qwen MTP verification by @YuhuaBillChen in #1986
- optize_qwen4exp by @JJJYmmm in #2037
- Support Apodex 1.1 checkpoints by @Lazarus-931 in #2019
- Add MTP support for Qwen3.8-Flash-Next by @lucasnewman in #2040
- Anthropic route: keep in-conversation system messages in place (fixes prefix-cache defeat) by @Lazarus-931 in #2052
- fix(server): transcribe m4a and other non-encodable audio uploads by @Lazarus-931 in #2051
- fix(rfdetr): honor --show-boxes instead of ignoring it by @Anai-Guo in #2024
- fix(sam3): let --threshold fall through to the documented per-task defaults by @Anai-Guo in #2025
- Fix positioned top-k sampling for speculative generation by @tjansn in #2029
- Add continuous batching support for Qwen3.8-Flash-Next by @lucasnewman in #2056
- fix(qwen3.8): support official FP8 conversion by @Thump604 in #2058
- Add APC disk caching for Qwen3.8-Flash-Next by @lucasnewman in #2059
- Fix streamed responses after tool calls by @lucasnewman in #2062
- Fix tool call parsing with quotes and nested values by @lucasnewman in #2064
- fix(attention): materialize a quantized KV state before hand-rolled attention (#2060) by @Anai-Guo in #2065
- Fix nemotron_h prefill crash when inputs and inputs_embeds are both passed by @Lazarus-931 in #2057
- Add pr contributor reminder by @Lazarus-931 in #2068
- Redesign APC for dense and hybrid model caches by @Blaizzy in #1960
- moe-offloading by @Lazarus-931 in #1813
- fix(cache): trim ChunkedKVCache on valid length, not buffer width (#2031) by @Anai-Guo in #2036
- fix(recurrent_gemma): initialize nn.Module in LanguageModel by @Anai-Guo in #2075
- Bump version to 0.7.0rc0 by @Blaizzy in #2083
- perf(qwen3.8): adapt MTP draft depth by @Thump604 in #2046
- feat(qwen3.8): add external PLE storage by @Thump604 in #2045
- handling issue #2073, cache search locally by @Lazarus-931 in #2076
- lfm2_vl: load flattened OptiQ checkpoints (#2088) by @Lazarus-931 in #2092
- fix(video): resolve frame sampling in one place (#2050) by @Lazarus-931 in #2100
- Support float-quantized compressed-tensors checkpoints by @Lazarus-931 in #2102
- diffusion_gemma: strip channel scaffolding from decoded output by @Lazarus-931 in #2101
- fix: require compatible mlx-audio for VoiceChat by @Thump604 in #2110
- fix: stop generation on the tokenizer's EOS when the config omits it by @Lazarus-931 in #2112
- Add LLaVA-OneVision by @Lazarus-931 in #2115
- Fix phantom KV in ragged batched speculative rollback by @Lazarus-931 in #2113
- Add DeepSeek-V4 DSpark speculative drafter by @Lazarus-931 in #2097
- Add LongCat-Flash-Lite-Sparse (LSA + n-gram) by @Lazarus-931 in #2063
- muse_glimmer: load flattened OptiQ checkpoints (#2108) by @Lazarus-931 in #2118
- fix(speculative): detect native MTP drafters via num_nextn_predict_layers by @Lazarus-931 in #2120
- fix(generate): chunk prefill for prompts shorter than prefill_step_size by @Lazarus-931 in #2119
- GLM-5.3 - 2026-08-28 by @Lazarus-931 in #2078
New Contributors
- @pyros-projects made their first contribution in #2034
- @YuhuaBillChen made their first contribution in #1986
- @tjansn made their first contribution in #2029
Full Changelog: v0.6.17...v0.7.0rc0