Qwen3.8-Flash-Next
The eighth family, and the first that arrives complete: engine, tool calling and vision together, rather than as follow-ups.
125B ordinary plus 51B hashed n-gram. 48 layers, as twelve blocks of three Gated DeltaNet and one QSA. 512 experts, top-10, plus a shared one. Resident 9.2 GiB; with cap 32, 16.5 GiB in total.
SNAP=/path/to/qwen38_flash_next_i4 ./qwen38 32
./coli chat --model /path/to/qwen38_flash_next_i4
The engines take the snapshot directory in SNAP and the per-layer cache as a positional argument. coli is the launcher and takes flags.
Tool calling. Qwen3.8 does not use the JSON tool dialect the other families speak. Its format is closer to XML:
<tool_call>
<function=get_weather>
<parameter=city>
Rome
</parameter>
</function>
</tool_call>
Handled end to end through the OpenAI-compatible server, so existing clients work unchanged.
Vision. Images work through the same server. Preprocessing is pinned against the official Qwen2-VL processor, including dynamic resolution: a 1080p frame becomes 2040 tokens. The tower is 27 blocks with learned positions bilinearly interpolated and 2D RoPE on top, verified token by token against the upstream reference.
Fixes
- Qwen3.6 int4 expert unpack is now vectorised. 2.10x faster CPU decode, bit-exact output. Thanks to @lornecodes.
- Qwen3.6 buffer widths and size_t casts corrected. Thanks to @Petsku01.
- Mirror stripe chunks are sized by measured bandwidth instead of split evenly, so a slow leg no longer holds back the fast ones. Thanks to @Unknown-Findout.
- A reasoning family was given the same output budget as a non-reasoning one, so on GLM-5.3, Inkling and Kimi the reasoning could consume the whole allowance before the answer began. The interactive budget is now 16384.
- The pilot prefetch worker could outlive the model it was reading from.
- A malformed container whose two config files disagree could overflow the heap in
load_meta.
Those last two were found because make clean was not removing two engines, which meant the sanitiser job had been re-running an old binary with no instrumentation in it. The job was green and empty. Both bugs predate this release.
Benchmarks
Twelve community hardware datapoints are now in the benchmark tables.
Known issues
#1278: GLM-5.3-Flash chat can end on thinking with --no-think. Open, cause not yet identified, not a regression in this release. If you hit it, that issue is the place to say so.
Upgrading
Nothing to change. Existing models, containers and flags keep working.