Inkling 2 Mini arrives as an experimental preview on its own engine, maya-ik2: the whole model in the VRAM of one 24 GB NVIDIA card on Linux, with thinking levels, tool calls and pictures.
Measured by @alesha-pro (Inkling 2 Mini, unsloth's UD-Q4_K_XL, ./maya.sh --bench, tokens/s):
| PC | Decode | Prompt reading |
|---|---|---|
| 1x RTX 3090 24 GB | 124.6 | 2,542 |
- Inkling 2 Mini, an experimental preview (#1095). Inkling 2 Mini (26 B) runs on a new engine,
maya-ik2, with every expert in the VRAM of one 24 GB NVIDIA card (RTX 30 / A100 and newer, Linux) and up to a 262,144-token context. Set it up with./maya.sh --setup --model inkling-2-minior from the Models page, where it is marked experimental; it downloads unsloth'sInkling-2-Mini-UD-Q4_K_XL(17.6 GB) and its vision tower (0.64 GB). - Everything you use with the other models. The OpenAI and Anthropic APIs and Maya's model API, streaming, tool calls, stop and the loop guard; greedy answers repeat across requests and starts. Its own defaults (temperature 1.0, top_p 1.0) and thinking from Off to Max (High by default).
- It reads pictures, on by default. The image encoder starts only when a picture arrives - on the GPU where there is room beside the model, otherwise on the CPU - so the model keeps its VRAM.
- Where the preview does not run (AMD, Windows, cards with less than 24 GB, GPUs older than the RTX 30 / A100, more than one GPU), the setup says why in one line. The other models build and run as before.
- GLM-5.3-Flash and Qwen pictures: the image encoder is compiled once more at your next setup (about 10-20 minutes, once); their pictures read the same.
- Credits. Inkling 2 Mini on
maya-ik2(#1095) was contributed by @alesha-pro, built on @mrsirg97-rgb's file format and reference forward pass (#981) and @OscarSheng's engine and server wiring (#1079), with @cobra91's Windows, CUDA 13 and two-GPU work in the branch. The setup, the Models page and the pictures are by @mw00 (peasantsmith).
Full list: CHANGELOG.md.
Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00