github mw00/project-maya v1.1.72

one hour ago

Inkling 2 Mini arrives as an experimental preview on its own engine, maya-ik2: the whole model in the VRAM of one 24 GB NVIDIA card on Linux, with thinking levels, tool calls and pictures.

Measured by @alesha-pro (Inkling 2 Mini, unsloth's UD-Q4_K_XL, ./maya.sh --bench, tokens/s):

PC Decode Prompt reading
1x RTX 3090 24 GB 124.6 2,542
  • Inkling 2 Mini, an experimental preview (#1095). Inkling 2 Mini (26 B) runs on a new engine, maya-ik2, with every expert in the VRAM of one 24 GB NVIDIA card (RTX 30 / A100 and newer, Linux) and up to a 262,144-token context. Set it up with ./maya.sh --setup --model inkling-2-mini or from the Models page, where it is marked experimental; it downloads unsloth's Inkling-2-Mini-UD-Q4_K_XL (17.6 GB) and its vision tower (0.64 GB).
  • Everything you use with the other models. The OpenAI and Anthropic APIs and Maya's model API, streaming, tool calls, stop and the loop guard; greedy answers repeat across requests and starts. Its own defaults (temperature 1.0, top_p 1.0) and thinking from Off to Max (High by default).
  • It reads pictures, on by default. The image encoder starts only when a picture arrives - on the GPU where there is room beside the model, otherwise on the CPU - so the model keeps its VRAM.
  • Where the preview does not run (AMD, Windows, cards with less than 24 GB, GPUs older than the RTX 30 / A100, more than one GPU), the setup says why in one line. The other models build and run as before.
  • GLM-5.3-Flash and Qwen pictures: the image encoder is compiled once more at your next setup (about 10-20 minutes, once); their pictures read the same.
  • Credits. Inkling 2 Mini on maya-ik2 (#1095) was contributed by @alesha-pro, built on @mrsirg97-rgb's file format and reference forward pass (#981) and @OscarSheng's engine and server wiring (#1079), with @cobra91's Windows, CUDA 13 and two-GPU work in the branch. The setup, the Models page and the pictures are by @mw00 (peasantsmith).

Full list: CHANGELOG.md.

Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.

Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00

Don't miss a new project-maya release

NewReleases is sending notifications on new releases.