NVIDIA Pascal cards as experimental, and unsloth's Q6_K_XL / Q8_0 GGUFs of GLM-5.3-Flash.
Update: the dashboard's About > Update, or git pull, then ./setup.sh (Windows: START-MAYA.bat). The engine recompiles; nothing is downloaded again.
What's new
- NVIDIA Pascal, experimental (#214 by @Jw88888, #212). The engine builds for compute capability 6.x - Tesla P40 / P100, GTX 10 - and the setup accepts these cards, with a note that Maya has not been validated on them yet. On Pascal the engine caches global loads in L1 (
-Xptxas -dlcm=ca), which its decode needs there: two Tesla P40s decode Maya-S24 at 28 tokens/s, with the same answers. Like the V100, Pascal needs CUDA 12.x. - unsloth's Q6_K_XL and Q8_0 GGUFs (#216 by @Xar1c). unsloth's UD-Q6_K_XL and Q8_0 quants of GLM-5.3-Flash keep some layers' routed experts at Q8_0, and the engine stopped at start with
unresolved weights. They run now.
NVIDIA GPUs of compute capability 7.0 and newer give v1.1.13's tokens.
Checked
- Two Tesla V100s: v1.1.13's tokens; the engine's GPU tests; the server end to end.
- GeForce GTX 1050 Ti (Pascal, the same chip generation as the P40): the engine builds for it by itself with the L1 caching, the engine's GPU tests pass, the setup accepts it as experimental, and Maya-S24 with twelve layers on it and the rest on a V100 gives the same tokens with and without the L1 caching.
- Radeon 8065S (Gorgon Halo, Windows 11): the engine builds.
- The engine builds on Linux and Windows; the GitHub checks.