github mw00/project-maya v1.0.11
Project Maya v1.0.11

latest releases: v1.1.74, v1.1.73, v1.1.72...
one month ago

Maya now runs on AMD GPUs (experimental): RX 7900 XT / XTX and Radeon AI PRO R9700 / RX 9070 on Linux, contributed by @boxwrench.

Update: git pull, then ./setup.sh. On AMD: ./maya.sh --backend hip --gpu 0 --check first (see docs/AMD_MAYA.md).

What's new

  • AMD (experimental, #7 by @boxwrench): ./maya.sh --backend hip builds Maya's engine with ROCm 7 for RX 7900 XT / XTX (gfx1100) and Radeon AI PRO R9700 / RX 9070 (gfx1201). Linux, one GPU, text only.
    • Measured by @boxwrench with Maya-S on the R9700: decode (writing the answer) about 20 tokens/s, prefill (reading the prompt) up to 490-560 tokens/s.
    • On the RX 7900 XT: prefill up to about 410 tokens/s.
    • Setup and measurements: docs/AMD_MAYA.md.
  • Prefill on AMD: the prompt projections go through hipBLASLt with tuning tables for each card, and the prompt runs in bigger sub-batches on cards with room (STRATA_GLM_PREFILL_SUB; NVIDIA keeps 256).
  • Earlier groundwork: the GPU-to-CPU signal fix came in v1.0.7 (#2).
  • NVIDIA: unchanged. The same tokens as v1.0.10 on 2x Tesla V100, and every target builds.
  • Not tested by us: we have no AMD hardware ourselves, so these results are the contributor's. Tell us how it runs on yours (--bench and --report learn AMD in a follow-up).

Don't miss a new project-maya release

NewReleases is sending notifications on new releases.