mw00/project-maya v1.1.50

on GitHub

Models you can download, swap and delete from the Models page or the API, a Turbo button, a thinking budget you can set, GLM Maya-XS, faster long prompts on DeepSeek, and a round of community pull requests.

• Models page: the catalog lists every Maya model with its size. Download one with a progress bar, load it, unload the running one, or delete one (a file that another model's folder links to stays). Nothing needs a command line.
• Model API: /api/tags, /api/ps, /api/show, /api/chat, /api/generate, /api/pull and /api/delete list, load, download and delete models from any app or script. A request for a model that is not loaded swaps to it, and keep_alive unloads it when idle. /api/ps shows "loading" while the weights load.
• Turbo: one button beside the model's parameters sets the fastest settings for the model (greedy sampling, no thinking, 32K context on GLM and MiniMax, expert skipping, the fastest drafting). It says "Turbo ON" with a flame while it is on, and switching it off restores your settings exactly. It trades some quality for speed; on the smallest quants (GLM and DeepSeek Maya-XS) it keeps every expert, so their answers stay sound.
• Thinking budget: a field in the settings sets how many tokens the model may think before it answers, and applies at once. The default stays 32768; GLM Maya-XS defaults to 8192.
• GLM-5.3-Flash Maya-XS: a new, smaller GLM quant (about 73 GB) for PCs with 64 GB of RAM and a 24 GB card at 128K context. The setup offers it as glm-maya-xs, with the int8 KV cache, a thinking budget of 8192 and sampling of temperature 0.7, top-p 0.95, min-p 0.05.
• DeepSeek long prompts: the prompt chunk now follows how many experts live in RAM. Where RAM holds most of them, a long prompt reads up to about 19% faster; where the VRAM holds them, nothing changes.
• DeepSeek thinking budget 0: the reasoning closes again, so answers and tool calls no longer land in reasoning_content (#786).
• Maya-S n-gram tables: Maya-S reuses Maya-XS's tables instead of downloading a second pair (#776).
• Log timestamps: every [maya] log line starts with the time, in the terminal and in the log file (#783).
• Gentoo: the HIP check finds versioned LLVM folders (#787, thanks @Wilsonfabozi).
• Community pull requests:
  • /tokenize and /detokenize endpoints (#764, thanks @cresslank).
  • A chat request's response_format reaches the model (#763, thanks @cresslank).
  • ignore_eos, and logit_bias on DeepSeek (#772, thanks @cresslank).
  • GLM prompts: the CPU lane takes a share where PCIe is not much slower than the CPU, with MAYA_GLM_PREFILL_CPU_GATE (#752, thanks @hugou74130).
  • DeepSeek on AMD keeps the router's weights in BF16 (#745, thanks @mrsirg97-rgb).
  • DeepSeek flow prompt chunks: an opt-in DMA/host all-reduce (#736, thanks @hajekt2).
  • Metal: no free-block exhaustion on long-context prefill (#742, thanks @appleguy).
  • Opt-in, off by default: peer-to-peer all-reduce over ReBAR for DeepSeek on 2+ GPUs, up to 29% faster prompts on the PCs it was measured on (DS4_AR_P2P=1, #762, thanks @486DX4-100). It was a little slower on V100s, so Maya does not turn it on.
  • Speed table and AMD docs (#756 thanks @Cayaf, #749 thanks @Diudid).

Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00