Strata 0.2.14: the prompt runs on Strata's own kernels on NVIDIA and AMD

The app and the Terminal are 0.2.14. Control and every engine package are 0.1.60. The update feed is sequence 125.
The Intel Arc engine stays at 0.1.59 in this release. The DGX Spark (linux-aarch64) engine stays at 0.1.59 too; its feed stays at sequence 123.

What changes for you

• Nothing to do. Update as usual (see Updating below). Your models, chats and settings stay.
• The engine reads your prompt with Strata's own code on NVIDIA and AMD cards, on Windows and Linux. Before, it used code taken from llama.cpp for this step. Answers can differ from 0.2.13 in some words; the tables below show how much.
• There are no new settings.

Highlights

• Prompt reading on Strata's own kernels on NVIDIA and AMD, Windows and Linux. A source build through setup builds the same way. -DSTRATA_MMQ=ON still builds the old llama.cpp path.
• Strix Halo (Windows), UD-Q4_K_XL, a 4,096-token prompt: 2,621 ms (0.2.13: 2,689 ms). An 8,192-token prompt: 4,910 ms (0.2.13: 5,091 ms).
• RTX 4090, IQ3_S, a 20,000-token prompt: 5,656 and 5,651 ms in two runs (0.2.13: 6,110 and 6,150 ms).
• RTX 5090, UD-Q4_K_XL, a 32,000-token prompt: 7,870 tokens per second (0.2.13: 6,851).
• Mac (M5 Max), IQ3_S, a 7,755-token prompt: 4,488 ms (0.2.13: 4,581 ms). Output, a 256-token story: 94.7 tokens per second (0.2.13: 92.8), the same text.

Speed against 0.2.13

Prompt times in milliseconds (ms, fewer is quicker) or prompt tokens per second, and output in tokens per second (more is quicker). Numbers in brackets are 0.2.13 on the same PC. Each row names the engine build it was measured on: most rows ran on builds of 11-14, and "build of 11-15" is the 0.1.60 build of that morning. The NVIDIA and AMD engines that ship were built once more after it, with a fix that changes only a model split across two or more cards; the Mac engine that ships is the build of 11-15.

Not measured against 0.2.13: RTX 5070 12 GB, RTX 3060 Ti 8 GB, Titan RTX, Tesla P100, two-card PCs, Radeon R9700 on Linux, Radeon 780M. On these the new engine was measured against the same version built with llama.cpp's kernels; see the details below.

How it was measured: the engine alone on the users' launch line, the same prompts in each arm, 2 or 3 interleaved rounds, a fresh engine start per round, greedy answers. Strix Halo 4K / 8K: 2 rounds, 3 prompts per size; the 0.2.13 figures come from a separate run on the same PC the same day. Mac: one launch per arm.

What changed for you

• Prompt reading. The prompt experts run on Strata's own kernels on NVIDIA and AMD, Windows and Linux. Setup's source builds build the same way. -DSTRATA_MMQ=ON still builds the old path.
• Still from llama.cpp / ggml after this release: its picture reader as the fallback and as a reader you can choose (Strata's own reader is the default where it was measured), the CPU expert kernels on the Mac, and stable-diffusion.cpp for the opt-in image preview. Intel Arc and the DGX Spark keep the 0.1.59 engine.
• Strix Halo (Windows): Path A is the default for the prompt's experts on UD-Q4_K_XL (43 of 48 layers). 4,096-token prompt: 2,623 ms with Path A, 2,739 ms without; 32,000 tokens: 19,151 ms with, 20,047 ms without. Output, all six kinds: 54.47 with, 54.72 without tokens per second; agent 60.86 with, 63.32 without; agent copy 91.64 with, 90.44 without. Quality, measured as KL against a run of the same engine with the prompt's experts in FP16 (lower is closer): 4,096 tokens 0.015488 with, 0.012877 without (ratio 1.2027, 95% range 1.0360 to 1.3995); 8,192 tokens ratio 0.9661 (0.8813 to 1.0563).
• Long prompts on one card are cut into pieces of equal size instead of full pieces plus a short rest. When an equal piece would be short, the engine keeps the old cut: with a prompt piece of 8,192 tokens or more, two equal pieces need 5,400 tokens each and three or more need 6,144 each. RTX 4090, IQ3_S, 8,279 tokens: 2,695 / 2,757 ms with this rule, 3,501 / 3,534 ms with equal pieces of 4,140. RTX 4090, IQ3_S, 11,450 tokens: 3,698 / 3,734 ms with two pieces of 5,725, 3,874 / 4,053 ms with the old cut. RTX 5070 12 GB, Q2_0, 11,000 tokens: 3,592 ms with two pieces of 5,500, 4,172 ms with the old cut.
• Two cards with a layer split: each card keeps only its own layers' dense weights (#4522), and the last card leaves room for the drafter's head (#4549). Two RTX A6000 held to 15.5 GB free each: output 75.88 tokens per second, against 64.16 before; 867 more expert slots on the first card.
• K/V streaming: unchanged from 0.2.13.
• Smaller changes, in a build without llama.cpp's kernels: Turing cards (RTX 20 series) read i-quant prompts on Strata's fused kernels; 8 GB cards read long prompts layer by layer with the fused kernels; the Radeon 780M on Linux reads its 16-bit prompt products in FP16; on Windows Strix Halo, prompt pieces under 1,024 tokens use the fused kernels' staged entry.

Fixed

• A model now starts without a drafter only when you choose Off. Setup downloads the drafter the start will use, and a start that finds its drafter missing uses the other one if it is on the PC, or downloads it first. The one exception is a PC without internet that has no drafter file yet: the model starts without one, says so, and the next launch with internet downloads it.
• Terminal on cards where Auto picks the MTP drafter (for example 12 GB cards with IQ3_S): Auto started without a drafter. Setup skipped the 5.2 GB draft layer, and the start then had none ("Drafting: off (no drafter is downloaded yet)"). 0.2.13 did the same. Now setup downloads the layer on the first launch. RTX 5070 12 GB, IQ3_S, a 31,823-token prompt: 20.7 tokens per second output without a drafter, 27.7 to 31.2 with MTP.
• On a PC without internet, a first launch on a card where Auto picks the MTP drafter stopped with "The model could not be installed." It now starts without the drafter.
• Mac: Auto with Q2_0 and the UD sets downloaded the 5.2 GB MTP layer although those sets run the DFlash2 drafter. It no longer does.
• Terminal: drafting Off was not always remembered. A later strata launch without a drafting word could start with the drafter again (seen on Strix Halo and an RTX 5070). The saved Off now holds, and strata launch --plan shows Off too.
• Terminal: drafting set to Off and then back to Auto started without a drafter. After one launch with drafting Off, strata launch --drafting auto kept starting the engine on the Off line (0.2.13.1 on an RTX PRO 5000 with IQ3_S: 139 tokens per second instead of 206). Choosing Auto, MTP or DFlash2 again now brings the drafter back.

Known problems

• Radeon on Linux (R9700): the driver pauses the card at times, for 115 to 810 ms in the starts of the engine that ships. The engine prints "the graphics driver paused the GPU for N ms". Count on 11-14, Qwen3.8 sets: 22 such lines in 25 starts of the engine that ships, 10 in 26 starts of the build with llama.cpp's kernels.
• Radeon R9700 on Linux: one engine start stalled on its first prompt and the driver reset the card (IQ3_S, 4 KiB memory pages). The engine stopped itself after 60 seconds without progress, as it is built to. We have not found the cause. It happened once on 11-14; the 51 starts in the pause count above ran.
• Radeon 780M on Linux: the GPU can reset on the first request after a start. The kernel log shows "MES failed to respond" and "GPU reset"; the reset recovers, and the engine stops itself after 60 seconds without progress. On the Radeon 780M class under Linux (gfx1103, also the 760M and 740M), 0.2.14 no longer registers the model's memory with the graphics driver; Strix Halo and the other integrated Radeon GPUs keep the 0.2.13 setting. The 780M on Linux is an experimental opt-in (STRATA_EXPERIMENTAL_GFX1103=1), and setup compiles its engine on the PC. Test PC (Radeon 780M, 64 GB, Linux kernel 7.0, Q2_0), first requests after a start on one fixed test line: with 0.2.14's setting 2 resets in 43 starts, both while memory was being swapped out during that request; with the memory registered as before, 4 resets in 7 starts of the build without llama.cpp's kernels, and 1 in 4 starts of the build with them (on a 2,048-token request). On 11-14, on the automatic cache size and with registered memory: 3 resets in 43 first requests without llama.cpp's kernels, 0 in 43 with them. 0.2.13 reset this PC twice in the same week, both on 8,192-token prompts. Not measured: 0.2.14's setting on the automatic cache size. Prompt times on the test line, 0.2.14's setting against registered memory: 84 to 256 tokens 2.85 to 3.07 s (1.50 to 1.92 s), 8,192 tokens 21.3 s (20.9 s); output 13.4 tokens per second in both.
• Not measured: UD-Q4_K_XL and the Coder model on the Macs; pictures with IQ3_S on the Mac; a full-precision reference for the Strix Halo Path A check; Strix Halo at 32,000 tokens on the build of 11-15; prompt rows against 0.2.13 on the cards listed under the speed table; the new engine on the DGX Spark with UD-Q4_K_XL (the reason its engine stays at 0.1.59); the app's first load on a card where Auto picks the MTP drafter (the Terminal was measured on an RTX 5070); whether the Linux AppImage opens again by itself after the in-app update from 0.2.13.
• Mac: choosing drafting Off downloads the 5.2 GB MTP layer on that launch, although the model then runs without a drafter as chosen (M5 Max, Q2_0: the launch took 272 s). A fix follows as an update through strata update.
• Peer mode (--peer-device, two cards) can fault on its first request, in about 4% of fresh starts. Not looked at again in 0.2.14.
• Radeon gfx1201: DFlash2 chosen by hand can stall on the prompt (11 of 50 engines on 0.2.11). Not looked at again in 0.2.14.
• The first launch of a fresh install takes several minutes while the model and its drafter are prepared: 450 s on an RTX 5070 with IQ3_S (it includes the 5.2 GB MTP download and about 5 minutes of preparing it), 301 s on Strix Halo in the Terminal (with the picture reader and the drafter downloads). Later launches took 59 to 75 s on these two PCs. 0.2.13: 398 to 406 s on the RTX 5070, 337 to 411 s on Strix Halo.
• The Terminal prints "Preparing the draft layer" at the end of the model's preparation even when no draft layer is downloaded or used. Nothing is fetched. A fix follows as an update through strata update.

Details: cards measured against the same version built with llama.cpp's kernels

Quality, forced-token KL against an FP16 prompt path (lower is closer): RTX 5070 Ti 16 GB, IQ3_S, all positions at 8,192 tokens: 0.00744 (with llama.cpp's kernels: 0.00787). RTX 4090, IQ3_S, first token at 8,279 and 16,884 tokens: 0.09812 with the prompt-cut rule, the same rows bit for bit as the build with llama.cpp's kernels.

Updating

• App: the in-app update offers 0.2.14, or download the installer below.
• Terminal: run strata update. From a fresh 0.2.13.1 install on a Linux PC (RTX PRO 5000 72 GB, IQ3_S), strata update took 36 to 37 s in two runs: engine 0.1.59 to 0.1.60, Control 0.1.59.1 to 0.1.60, and the new engine started and answered one token before the update finished. The launch profile and settings files were byte-identical before and after. strata components rollback went back to the 0.1.59 engine and a later strata update moved forward again with nothing downloaded; a launch with the DFlash2 drafter and chats passed after each step.
• Intel Arc (Linux) and the DGX Spark stay at 0.1.59. strata update keeps the Intel Arc engine from 0.2.13. The DGX Spark keeps its 0.1.59 engine and Control, and its update feed stays the 0.2.13 one.

What we ran on the release files themselves (this release's own feed and installers, checked with the key built into Strata):

On the first three of these PCs, drafting Off stayed off after a restart, and Auto brought the drafter back.

• The Windows installer: installed as a standard user (no administrator) in 16 s on Strix Halo, and its self-test passed 13 of 13 steps. Uninstalling removed the program and its entry.
• Mac: the app, the dmg and the Terminal are signed with our Developer ID (80 files in the app, 66 in the Terminal) and notarised by Apple ("Accepted"). The app's self-test passed 12 of 12 steps. No model was run on the final Mac files.

Downloads

36 files, 8,554,091,799 bytes in all. Names, sizes and SHA-256 are in SHA256SUMS.txt.

• App installers for Windows (Strata-0.2.14-windows-x64-setup.exe), Linux (Strata-0.2.14-linux-x86_64.AppImage) and the Mac (Strata-0.2.14-macos-arm64.dmg), with .sig files for the in-app update (Strata.app.tar.gz for the Mac).
• Control 0.1.60 for Windows, Linux and the Mac; Control 0.1.59 for the DGX Spark (the same file as in 0.2.13).
• Engines 0.1.60:
  • Windows: CUDA 13, CUDA 12, AMD RX 9000, RX 7000 / Strix Halo, RX 6000.
  • Linux: CUDA 13, CUDA 12, AMD RX 9000, RX 7900, Strix Halo, RX 6000.
  • Mac: Metal.
• The DGX Spark engine and its DeepSeek engine, 0.1.59 (the same files as in 0.2.13).
• Two drafter files, the same as in 0.2.13.
• The feeds, latest.json, install.sh, install.ps1, install-manifest.txt and SHA256SUMS.txt.

The Intel Arc engine and the DeepSeek engines for Windows, Linux and the Mac stay at 0.1.59; the update feed downloads them from the 0.2.13 release.

Thanks

• cobra91 for the two-card layer split work (#4522, #4549), built on Hardin22's Strata-DualGPU.

Strata is free and open source. If it runs well on your PC, a coffee keeps the work on it going: