DeepSeek-V4.1-Flash reads long prompts up to ~1.5x faster with the same answers (+30% on a 32K prompt on one GPU, +43-54% on 8-16K prompts on a Strix Halo), its prompts follow DeepSeek's own format for agents and tools, and a round of community work.
Update: the dashboard's About > Update, or git pull, then ./maya.sh (Windows: START-MAYA.bat). An install of v1.1.18 keeps its files and works as it is.
What's new
- DeepSeek: long prompts up to ~1.5x faster, same answers. DeepSeek-V4.1-Flash's layers 20-39 see a prompt's
earlier tokens only through what layer 20 keeps of them, so they need their full work only on the last ~2,700 tokens
of a prompt. A long prompt now skips the rest: on one GPU, prompts of ~3,700 new tokens or more are read faster -
the longer the prompt, the more it saves (+30% on a 32K prompt on one GPU, +43-54% on 8-16K prompts on a Strix
Halo) - with the same answers.MAYA_DS4_PROMPT_SKIP=2goes further, as
DeepSeek's own serving does (the last 128 tokens only; the answers shift a little);0turns it off. Learned from
DeepSeek's DeepInfer. - DeepSeek: prompts in DeepSeek's reference format (#300 by @Gazelle221B, fixes #299). Conversations are rendered
with DeepSeek's own encoder: later system messages, message-local tools, parallel tool results and their call ids
are kept, so agent sessions see what they sent. - DeepSeek: the engine ends at QUIT (#306 by @loktar00), so stopping or swapping models no longer waits out a
timeout; profiling (DS4_PROF) on several GPUs no longer disturbs inference (#307 by @cosmicnag). - EXL3: exllamav3 1.6's half-integer bitrates (K 1.5 / 2.5 / 3.5) for self-calibrated community quants, every
decoder bit for bit exllamav3's (#265 by @sociolog). - GLM: the recurrent-state backup for drafting on a multi-GPU NextN split (#290), a warm-up prompt before READY,
opt-in (MAYA_GLM_WARM_PROMPT, #285), and a docs fix (#287), all by @gopinath87607; the gfx1100 hipBLASLt table for
ROCm 10 - faster prompts on RX 7900 cards with ROCm 10 (#292 by @Cass67). - README: the community's speed reports since v1.1.12 in the speed table.
Checked
- DeepSeek-V4.1-Flash: the long-prompt skip against the full work - the prompt's last logits within rounding (KL 1.5e-4) and the same greedy answers, on NVIDIA and AMD; a live agent conversation with a tool call through the server; the GGUF kernel tests.
- EXL3: the kernel tests, every half-integer decoder bit for bit exllamav3's.
- Builds: Linux CUDA, Windows CUDA (MSVC), Windows HIP (Radeon 8065S); the setup's and the server's tests.