Faster prompts with the INT8 cache, Windows fixes for speed in the background and for the RAM tier, and a guard against model files that change after setup.
Update: the dashboard's About > Update, or git pull, then ./setup.sh (Windows: START-MAYA.bat). The engine recompiles; nothing is downloaded again.
What's new
- Windows: full speed with Maya's window in the background. The engine opts out of Windows' power throttling (EcoQoS) itself, which moved its CPU lane onto a hybrid Intel CPU's efficiency cores whenever every Maya window was in the background - decode dropped to a fraction of its focused speed (#114, thanks to @githug2024 for testing
powercfg). - Windows: the RAM tier fills the pinning limit. Where Windows refuses to pin a planned tier, the engine now measures how much still pins and sizes the tier to it, instead of stepping down 3% at a time - those steps left a 128 GB PC's tier 2.5 GB under its limit, and the SSD served those experts (#129, measured by @anbow1). With several expert sizes each takes its planned share. The log's "RAM tier sits on node X of N seen" line now appears only on PCs with two NUMA nodes or more, where it means something (#114, #129, #147).
- A guard against model files that change after setup (#155, traced by @zhanghuqqqq). Before the setup builds a pack from a Maya model's files, it checks them against their published sha256 - files it downloaded were always checked, files already in the folder now are too, once each. The pack records the files it was built from, and a start refuses it if they changed afterwards (
./maya.sh --setup --repackbuilds it again): a pack built while a download manager was still filling files it had created at full size answered garbage. - Prompts with the INT8 cache (
--kv int8). The prompt attention converts the cache's INT8 records to FP16 for less (a byte permute and FP16 arithmetic instead of int-to-float conversions), with the same bits as before - checked for every code and every scale on NVIDIA and AMD. On two V100s the attention's extra time over FP16 went from +18% to +11%, and INT8 prompts are now within about 2% of FP16's (8K-32K).
Checked
- The same tokens as v1.1.6 on two V100s - the default cache with the CPU lane on and off, and INT8 - and on an AMD Radeon 8060S under Windows (default cache and INT8).
- The INT8 records' conversion bit for bit against the old one for every code and every finite scale, on a V100 and a Radeon 8060S.
- The RAM tier at a simulated pinning limit (Linux,
STRATA_GLM_PIN_CAP_MB): with 8.8 and 6.3 GB limits under a 12 GB plan it pinned 8.1 and 5.6 GB - the limit less the 0.5 GB kept free and under 0.25 GB of rounding. The Windows PCs from #114 and #129 will show it on a real limit. - The pack check's unit tests; the engine builds on Linux, Windows (MSVC) and Windows + AMD and starts with the power-throttling opt-out; the server end to end on two GPUs; the GitHub checks.