github mw00/project-maya v1.1.67

57 minutes ago

A correctness fix for GLM on PCs where the experts do not all fit in VRAM, GLM now starts on a small graphics card instead of refusing, a fix for fresh installs on Windows, a little more GLM speed on the AMD + NVIDIA pair, and more exact wording about drafting.

  • Fix: GLM could compute an expert with another expert's weights: on a PC where some of GLM's experts sit in RAM, Maya moves experts between RAM and VRAM while it writes. An expert could be moved into VRAM twice at the same moment; once both copies had left again, Maya could still look for it in a place in RAM that by then held a different expert, and used that one's weights under its name. There was no error, only answers that were slightly off, and how often it happened grew with the share of the experts kept in RAM. An expert is now in VRAM only once, and a place in RAM is never reused while anything still points at it. Who is affected: GLM (GLM-5.3-Flash and GLM-5.3) on PCs where part of the experts sit in RAM; a PC whose VRAM holds every expert is not. On an affected PC answers can differ from v1.1.66, because they are now correct. Speed is unchanged.
  • GLM decode on the AMD APU + NVIDIA GPU pair, about 4% more: about +4% decode on top of v1.1.66 with a Tesla V100 beside the APU, measured as an A/B on the same PC (GLM-5.3-Flash Maya-S), with the same answers. Between the NVIDIA card's calls the integrated GPU had nothing to do and its clock dropped; Maya now keeps the integrated GPU busy with a tiny task while an answer is written, so its clock stays up. This costs power while an answer is written: measured about 2 W more on the APU and about 2 W more on the NVIDIA card (it waits less), about 4 W for the pair. It stops when the answer ends, and the energy per token is about 2% lower, because the answer is finished sooner. Maya times both ways during the first answers and keeps the integrated GPU busy only where that is faster on your PC. MAYA_GLM_HELPER_WARM=0 turns it off.
  • What drafting guarantees, said exactly: with drafting (the MTP block or a DFlash2 drafter) the model checks every drafted token, so Maya only writes tokens the model itself would pick. The docs and the setup text said "the same answers" as without drafting, which was too strong: a drafter takes VRAM, so on a PC where the experts do not all fit, other experts sit in VRAM, and answers can differ from a run without the drafter. The docs and the setup text now say this.
  • Fix: the setup on Windows could stop while unpacking llama.cpp: from a folder such as C:\Users\<name>\project-maya with a name of six letters or more, one file's path passed Windows' limit of 260 characters during the unpack, and the setup ended with "No such file or directory". The unpack no longer depends on that limit, so a fresh install works whether or not Windows' long paths are switched on.
  • Setup: the model menu: GLM-5.3 (744B) has a row of its own, after GLM-5.3-Flash, and each of the two rows then lists that model's files only. The MiniMax and Qwen rows no longer say "new", and the Qwen row is the model's name alone (Qwen-3.8-Flash-Next); its files are named on the next screen as before.
  • GLM starts on a small graphics card instead of refusing: on a card with too little VRAM left for GLM's experts (for example GLM-5.3-Flash on an 8 GB card that also drives the desktop), Maya stopped at start with "no room for the expert pool". It now starts anyway: the experts stay in RAM and are computed on the CPU, or sent to the card just for the token that needs them. The answers are the model's as before; writing and reading prompts are much slower than on a card that holds the experts, drafting is off there, and the start's log says so. One GPU only; PCs that started before start and run exactly as before.
  • Setup: said before the download when GLM-5.3-Flash will be slow on your card: on a PC with one NVIDIA card, the setup now works out whether the card can hold the quant you pick: the weights the card always holds, the context, the engine's working memory and the smallest expert pool, against the card's VRAM less what the desktop and other programs use on it now. Where the experts do not fit (an 8 GB card that also drives the desktop), the model menu marks the row "runs slowly on this PC's 8 GB card (experts on the CPU)", the setup says so before the download and names the quants that do fit, and it goes on. Where the card is too small even for the weights it always holds, the row says "does not fit" and the setup stops before any download (--force-fit sets it up anyway). Two GPUs or more, AMD and Mac are not judged, as before.
  • For developers: MAYA_GLM_TIER_VERIFY=1 (off by default, a diagnostic) checks GLM's expert tables after every token and reports an entry that points at the wrong place. It only reports.

Full list: CHANGELOG.md.

Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.

Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00

Don't miss a new project-maya release

NewReleases is sending notifications on new releases.