What's Changed
- Add support for TTS and STT endpoints in the server by @lucasnewman in #1358
- Add prefill policy support for diffusion models by @lucasnewman in #1357
- Fix diffusion mxfp4 embedding dequantization by @Blaizzy in #1356
- fix: guard cache_offset == 0 against array truth-value error in continuous batching by @imi4u36d in #1354
- Fix DiffusionGemma generation from the server by @lucasnewman in #1368
- Improve diffusion unmasking visualization by @Blaizzy in #1361
- [Cohere] Fix streaming thinking parsing after prompt-opened by @Terrencezzj in #1369
- [Fix] interleaved reasoning by @Terrencezzj in #1370
- Fix color shift from tiled VAE in Flux.2 models by @lucasnewman in #1381
- Revert "[Cohere] Fix streaming thinking parsing after prompt-opened" by @Blaizzy in #1389
- Fix Gemma 4 namespaced tool calls by @Blaizzy in #1379
- Forward strict flag when loading weights by @Blaizzy in #1345
- Add optional API key for the server by @lucasnewman in #1385
- Fix SFT masks for Gemma4 unified by @kqw8 in #1392
- Add preloading for multiple model kinds in the server by @lucasnewman in #1372
- fixes #1404: wire --prefill-step-size into BatchGenerator constructor by @royteeuwen in #1405
- fixes #1402: Every first request after server startup causes a full model reload by @royteeuwen in #1403
- fixes #1406: move draft-kind/draft-block-size env vars outside draft-model guard by @royteeuwen in #1407
- fix(StoppingCriteria): copy eos_token_ids to prevent caller-list aliasing by @raullenchai in #1410
- [performance] Optimize PaddleOCR-VL vision fast paths by @scyyh11 in #1396
- Avoid clearing MLX cache from stream finalizers by @Blaizzy in #1412
- Add Moondream2 model support by @Abdullah-Alharbi61 in #1395
- Qwen3.5 on CUDA: fix Metal-only crashes + chunked gated-delta prefill (8.5x faster prefill) by @Blaizzy in #1423
- Add reasoning_content to chat messages by @Blaizzy in #1411
- improved turbo quant's prefill by @Lazarus-931 in #1433
- adding hisa to dsa based models(currently only deepseek) by @Lazarus-931 in #1434
- fix for #1446 by @Lazarus-931 in #1447
- Fix Qwen3.5 MoE gate-up sanitization by @Blaizzy in #1450
- Fix Gemma 4 cached vision mask overlay by @Blaizzy in #1445
- Fix stale GenerationBatch processor state by @Blaizzy in #1451
- kimi_k25: 3D MoonViT video support (Fixes #1436) by @Lazarus-931 in #1438
- Fix TurboQuant MTP rollback tail zeroing by @Blaizzy in #1432
- convert: operate on the inner model for text-only wrappers (Fixes #1437) by @Lazarus-931 in #1439
- Fix Qwen3.5 TurboQuant batch rollback trim by @Blaizzy in #1453
- Fix quantized Diffusion Gemma sampler speed by @Blaizzy in #1359
- Remove video generate command by @Blaizzy in #1454
- fix(StoppingCriteria): copy eos_token_ids in reset() to prevent caller-list aliasing by @SuperMarioYL in #1430
- Guard empty MRoPE position batches by @Blaizzy in #1455
- Improve Responses API reasoning and tool stream events by @justin-schroeder in #1458
- Select weight files via model.safetensors.index.json when present(fix #1329) by @Lazarus-931 in #1464
- fix(apc): enable APC prefix cache with MTP speculative decoding by @vedang in #1463
- fix(deepseek_v4): clamp shared-expert SwiGLU to swiglu_limit by @popfido in #1469
- Fix Gemma 4 bare namespaced tool calls by @Blaizzy in #1473
- fix(qwen3_5_moe): support split expert weights for ornith 1.0 35b by @scaryrawr in #1472
- Minimax m3 support by @ivanfioravanti in #1374
- Add Unlimited-OCR model support by @shuuul in #1427
- fix(qwen3-omni): eval mRoPE position_ids before the thinker forward by @yuhuanowo in #1461
- fix(rope_utils): make M-RoPE apply differentiable for training by @Lazarus-931 in #1475
- Fix Qwen3-Omni repeated generation by @Blaizzy in #1476
- Fix DiffusionGemma tool marker tokenization by @Blaizzy in #1477
- Constrain transformers below 5.13 by @Blaizzy in #1488
- fix #1480 by @Lazarus-931 in #1484
- Make Qwen MRoPE request-local under continuous batching by @Blaizzy in #1486
- fix #1481: quantized KV cache crashes in mask-length checks by @Lazarus-931 in #1483
- Reject empty OpenAI server inputs by @Blaizzy in #1491
- Consolidate caches + RoPE handling by @lucasnewman in #1494
- fix(lfm2_vl): only create projector LayerNorm when projector_use_laye… by @Lazarus-931 in #1493
- Bump version to 0.6.4 by @Blaizzy in #1499
- Fix MolmoPoint EOS config by @Blaizzy in #1500
- Fix Phi-3-V chat stop tokens by @Blaizzy in #1501
- Always sanitize loaded weights and simplify audio preprocessing by @Blaizzy in #1498
- Add torch-free Pixtral processor by @Blaizzy in #1502
- Support Gemma 4 variable soft-token budgets by @Blaizzy in #1503
- Fix DiffusionGemma processor and video inputs end-to-end by @Blaizzy in #1492
- Fix MiniCPM-V 4.6 weight sanitization by @Blaizzy in #1506
- Use model type in diffusion text-only error by @Blaizzy in #1507
- Unify diffusion generation path by @Blaizzy in #1508
- migration from mlx-lm dependency by @Lazarus-931 in #1509
- Remove obsolete test mocks for safetensors by @lucasnewman in #1511
- Port glm_moe_dsa language model by @Blaizzy in #1390
- Fix computer use action parsing by @Blaizzy in #1514
- Add qwen2 model type and remove mlx-lm from fastvlm by @Lazarus-931 in #1516
New Contributors
- @imi4u36d made their first contribution in #1354
- @kqw8 made their first contribution in #1392
- @royteeuwen made their first contribution in #1405
- @raullenchai made their first contribution in #1410
- @Abdullah-Alharbi61 made their first contribution in #1395
- @Lazarus-931 made their first contribution in #1433
- @justin-schroeder made their first contribution in #1458
- @vedang made their first contribution in #1463
- @scaryrawr made their first contribution in #1472
- @shuuul made their first contribution in #1427
Full Changelog: v0.6.3...v0.6.4