Hy4-Flash decodes faster where the CPU computes experts and on Radeon, reads long prompts faster on Radeon, runs faster on busy PCs and reads its first prompt at full speed. It also picks the fastest drafting for each PC, can keep a long context's cache on an SSD, and there are fixes for GLM, DeepSeek, Hy4-Flash and the setup.
Measured on our test PCs (Hy4-Flash Maya-XS, v1.1.70 -> v1.1.71, tokens/s):
| PC | Decode (./maya.sh --bench)
| Prompt reading, 8K | Prompt reading, 32K |
|---|---|---|---|
| 1x Tesla V100 32 GB + Xeon Gold 6138 (the CPU computes the experts that don't fit) | 41.0 -> 53.1 (+30%) | 630 -> 630 | 580 -> 588 (+1%) |
| Radeon 8065S (Gorgon Halo) | 40.9 -> 43.7 (+7%) | 421 -> 422 | 373 -> 430 (+15%) |
The first start after installing or updating measures your PC once (about 1-4 minutes, depending on the PC); the setup and the server say so before it starts, and the next starts skip it.
- Faster Hy4-Flash decoding where the CPU computes experts. The experts kept in VRAM now follow the ones your answers use, and the CPU side no longer stalls while other programs keep the PC busy (#1082). Decode against v1.1.70: about +30% on one Tesla V100 with the CPU computing the rest and +7% on a Radeon 8065S (Gorgon Halo).
HY4_HOT=0turns the expert following off. - Faster long prompts on Radeon. Hy4-Flash reads 32K-token prompts about 15% faster on a Radeon 8065S (RDNA 3 / 3.5 cards in general).
- The first prompt reads at full speed. The first start measures the PC before it says ready, so the first prompt no longer reads slowly while the PC is being measured.
- Repeatable on several GPUs. The first answers match their repeats and the same start gives the same prompt route on one GPU, the layer split and tensor parallel.
- The fastest drafting for each PC (
--drafting auto). The first start tries the drafting choices and keeps the fastest for that PC;--speed-calibrationmeasures again, and--drafting off|on|mtp|dfly2still picks by hand. On an 8 GB card--mtpno longer stops the start. - A long context's cache on an SSD (
--kv-spill auto). Where the RAM cannot hold a long context, the older positions go to a file on a drive (--kv-spill-dir).HY4_KV_MORE=1(opt-in) streams the cache also where it fits but the freed VRAM holds more experts. - Windows with several NVIDIA cards. The VRAM Windows keeps for itself is measured for each card, and
--room-reportshows the cards side by side. - Opt-ins.
"web_ui": falsein the config (orMAYA_WEB_UI=0) serves the API alone, with no web page and no browser opening at the start (#1089);--gpu-fetch/HY4_FETCHlets the GPU fetch the experts it needs from RAM itself,autotiming the options once. - Fix: GLM-5.3 with drafting no longer stalls (#860). A regression since v1.1.53 that could end in an error is gone.
- Fix: DeepSeek no longer hangs on two AMD GPUs (#1086, #1027).
--no-expert-cacheis no longer needed. - Fix: DeepSeek after a refused unbuffered read (#1076). The file keeps being read unbuffered, and the refusal is logged once.
- Fix: Hy4-Flash after an agent client rewinds a chat (#1091). Wrong answers after rewinds to different depths are gone (#1092).
- Fix: running the setup again keeps every setting (#1089). Your GPUs, port, host, API key, model management, idle unload, loop guard and the rest of the server settings stay;
--port/--host/--api-keygiven to the setup win, and a copy of each changed config is kept. - Fix:
--rebuildrebuilds every engine the PC uses, not only the first model's. - Fix: AMD read-only model folders (#1090). The start says once that the GEMM timings cannot be kept, and
HY4_LT_STORE=<file>keeps them elsewhere. - DeepSeek: the CPU side's copies stay on their own NUMA node on Linux PCs with two nodes or more (#1083).
- Credits. The CPU-side fix for busy PCs (#1082) was contributed by @lastbl0c, the DeepSeek NUMA copies (#1083) by @needmorevram, the AMD GEMM timings fix (#1090) by @Antero128 and the fix for Hy4-Flash after rewinds (#1092) by @lij768423-svg. The rest of this release is by @mw00 (peasantsmith).
Full list: CHANGELOG.md.
Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00