Maya now runs on AMD GPUs (experimental): RX 7900 XT / XTX and Radeon AI PRO R9700 / RX 9070 on Linux, contributed by @boxwrench.
Update: git pull, then ./setup.sh. On AMD: ./maya.sh --backend hip --gpu 0 --check first (see docs/AMD_MAYA.md).
What's new
- AMD (experimental, #7 by @boxwrench):
./maya.sh --backend hipbuilds Maya's engine with ROCm 7 for RX 7900 XT / XTX (gfx1100) and Radeon AI PRO R9700 / RX 9070 (gfx1201). Linux, one GPU, text only.- Measured by @boxwrench with Maya-S on the R9700: decode (writing the answer) about 20 tokens/s, prefill (reading the prompt) up to 490-560 tokens/s.
- On the RX 7900 XT: prefill up to about 410 tokens/s.
- Setup and measurements: docs/AMD_MAYA.md.
- Prefill on AMD: the prompt projections go through hipBLASLt with tuning tables for each card, and the prompt runs in bigger sub-batches on cards with room (
STRATA_GLM_PREFILL_SUB; NVIDIA keeps 256). - Earlier groundwork: the GPU-to-CPU signal fix came in v1.0.7 (#2).
- NVIDIA: unchanged. The same tokens as v1.0.10 on 2x Tesla V100, and every target builds.
- Not tested by us: we have no AMD hardware ourselves, so these results are the contributor's. Tell us how it runs on yours (
--benchand--reportlearn AMD in a follow-up).