A new model: Hy4-Flash, Tencent's 220B model, runs in Maya with four Maya quants of our own, on NVIDIA cards from 8 GB of VRAM and on AMD under Linux.
-
New model: Hy4-Flash, in four Maya sizes. Tencent's Hy4-flash (220B parameters, Tencent Hy License) is now a model in Maya. We quantized it ourselves into four sizes, all on Hugging Face, each measured against the full-precision model on held-out conversations:
Size Model file Same top token as the original KL divergence RAM + VRAM Maya-XS 44.8 GB 88.5% 0.16 56-64 GB Maya-S 53.0 GB 91.8% 0.072 68-80 GB Maya-M 68.4 GB 93.6% 0.046 80-96 GB Maya-L 89.5 GB 96.6% 0.012 104-128 GB Every size also uses the model's n-gram table (63.4 GB, one file shared by all sizes). It is downloaded once, stays on the SSD and is read from there while the model answers, so it does not have to fit in memory. Keep it on an SSD / NVMe, not a hard disk.
-
Setup and model picker. Pick "Hy4-Flash" in the setup's model list, or
./maya.sh --setup --model hy4-flash-maya-xs(Windows:START-MAYA.bat --setup --model hy4-flash-maya-xs). Before any download the setup works out which size, which file and which context fit your PC (4K up to the model's 1M, each offered where it fits the VRAM). What does not fit the graphics card is computed on the CPU from RAM, and read from the SSD when RAM is short. Maya-XS runs on cards from 8 GB (the setup picks a file made for small cards by itself); Maya-S, Maya-M and Maya-L need a card of about 12 GB or more. -
Drafting. Maya's sizes draft with Tencent's MTP block: it guesses the next tokens and the model checks every one, so Maya only writes tokens the model itself would pick. The block is one extra file for every size (7.5 GB, downloaded once). It takes about 4 GB of VRAM, so the setup turns drafting on where that fits (cards from about 16 GB);
--drafting off, or Drafting on the Models page, turns it off. Where every expert sits in VRAM, drafting gave about +57% decode on our test rig; with part of the experts on the CPU, about +10% to +30%. -
Prompt reading. The engine picks the largest prompt chunk that fits by itself at every start: about +30% prompt speed on a multi-GPU rig and about +70% on one card against the engine's earlier default. Where NVIDIA cards hold the whole model, prompts beyond 8K tokens keep the fast path up to 16K or 24K tokens, picked by the engine from the free VRAM.
-
Pictures. Hy4-Flash reads pictures with its own vision tower. The setup downloads the vision file with the model (0.8 GB;
--no-visionskips it), and Images on the Models page turns pictures on or off. On cards with little free VRAM, big pictures are made smaller first. -
The model's own defaults and thinking. Tencent's sampling (temperature 0.9, top-p 1.0), the release's chat template, and its two thinking modes: High (the default) and Off. A client that asks for another level gets High. Tool calls, streaming and the loop guard work as for every model.
-
Runs on. NVIDIA GPUs from the Tesla V100 on, under Linux and Windows, on one card or several (equal cards share every layer). AMD under Linux, including Strix Halo and Gorgon Halo PCs with no separate graphics card: a 64 GB Halo holds Maya-XS whole, 96 GB Maya-M, 128 GB Maya-L.
-
Dashboard. Hy4-Flash has its entry on the Models page (load, unload, delete; context, GPUs, images, drafting, prompt chunk), the chat offers High and Off, and the Monitor shows the experts computed on the GPU against those on the CPU.
-
Tencent's own Q4_K_M. The same setup also offers Tencent's own Q4_K_M files (AngelSlim's GGUF), listed after Maya's sizes, or
--model hy4-flash-q4_k_m, with drafting and a context up to 256K. It needs about 170 GB of RAM + VRAM. -
Known limits in this release.
- One request at a time; concurrency is coming soon.
- On AMD it is slower than DeepSeek on the same chip for now (prompt reading most of all); AMD tuning comes next. AMD on Windows and Mac are not supported yet.
- On AMD, with Maya-M and Maya-L an answer with drafting can differ from the same answer without it; turn drafting off if you need them equal. Maya-XS and Maya-S give the same answer either way.
- No drafting on 8 GB cards (the block does not fit), and pictures are made smaller there.
- Prompt reading slows down on very long prompts (tens of thousands of tokens); more work on it is planned.
- With greedy decoding (temperature 0) a long thinking phase can start repeating itself, as it can with the original model; keep the recommended sampling.
-
Credits. The base NVIDIA engine for Hy4-Flash (
maya-hy4) and vision support were contributed by @needmorevram (#1040, #1051). The AMD device code is from @Cass67. The Maya quants, the setup and model picker, drafting for Maya's sizes, small-card and Windows support, AMD and Halo support, the prompt-reading work and the dashboard integration are by @mw00 (peasantsmith).
Full list: CHANGELOG.md.
Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00