DeepSeek-V4.1-Flash now runs unsloth's DeepSeek GGUF quants, and the setup lists UD-IQ3_S: close to Q8's quality for PCs
with about 270 GB of VRAM + RAM. Long agent conversations are no longer read again from the start when the loop guard
steps in (the next turn came back about 15x sooner in our tests), answers that write a turn-end marker inside their
thinking now finish, and the Monitor shows which drafter runs and how many of its drafts the model keeps, live. Maya
started as a service skips the speed calibration instead of looping, and experts read from the SSD use as many reads at
once as the drive serves fastest. Against v1.1.42 on our test PCs, decode and prompt speeds
are unchanged for every model.
- DeepSeek: unsloth's GGUF quants (thanks to unsloth for the quants):
- UD-IQ3_S is in the setup's list, between Maya-S and the Q8 (
--ds4-quant gguf-iq3_s). It measures close to
Q8: a KLD of 0.061 against the original model, where Q8 has 0.047 and Maya-S 0.119 (KLD: how far the quant's
next-token choices drift from the original's; lower is better). About 262 GB of VRAM + RAM hold it, plus the
161 GB of n-gram tables read from the SSD. - The other unsloth quants can be set up by hand:
--ds4-quant gguf-iq1_s,gguf-iq1_m,gguf-iq2_xxs,
gguf-q2_k_xl,gguf-iq3_xxsandgguf-q3_k_xl, pinned to unsloth's 2026-10-28 upload; UD-Q4_K_XL and
UD-Q8_K_XL work as before. - Pictures with every unsloth quant, from the picture encoder file in unsloth's own repo (a 0.97 GB download).
- One copy of the n-gram tables: UD-IQ3_S and Maya's quants hold the same 161 GB of tables, so with one of them on
the disk the setup of the other reuses that copy instead of downloading it again. - The setup's free-space check counts each file against the disk it actually lands on.
- UD-IQ3_S is in the setup's list, between Maya-S and the Q8 (
- Agents and the loop guard:
- A long conversation's prompt is kept whole when the loop guard steps in, so the conversation cache is still used:
on a 150K-token agent conversation the next turn took about 15x less time with GLM and with DeepSeek (#704, thanks
@tonykee; fixes #702).MAYA_LOOP_GUARD_COLLAPSE_MAXsets the size from which a prompt is left whole. - The loop guard's follow-up call always fits the context: an answer it lets go on, or a request without
max_tokens, no longer ends in an engine error (a v1.1.41 regression; #696, thanks @Jellepepe). - A turn-end marker written inside the thinking is now read as the model's text: the answer goes on and finishes
instead of stopping mid-thinking, with DeepSeek and GLM, and a client's own stop strings work inside code again
(#709, thanks @markrussi).
- A long conversation's prompt is kept whole when the loop guard steps in, so the conversation cache is still used:
- Monitor: the Speculation card shows the drafter that runs (DSpark, DFlash2, MTP, prompt lookup) and the share of
its drafts the model keeps, live while an answer is written, for GLM, DeepSeek, MiniMax and Qwen. "SSD only" in
"Where the experts live" now shows its GB, and the Throughput card has no blank space on a phone any more. - Running Maya as a service: a start with no terminal (systemd, a script, or
--yes) begins with the default
settings instead of measuring first, and prints the one command to run the calibration later. A service no longer
loops for ever when its start timeout ends the calibration. docs/SETUP.md has a systemd section (#690, thanks
@ZackO2o for the report). - Experts read from the SSD (Linux): how many reads run at once is now measured for each drive at its first start (about
a second, then remembered), instead of guessed from the CPU. On an NVMe beside an 8 GB card, Qwen CYBER-FROST
decoded up to 5% faster with its experts read from the SSD.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00