Niko1221/Strata v0.1.9

Strata v0.1.9

on GitHub

No more stalls on a full GPU, access from other devices on your network, and an experimental option.

• Fixed: requests could stall forever at large contexts. The engine sized its expert cache before loading the output head (~0.5 GB), so the head used up the VRAM it meant to keep free. IQ3_S at 128K ended with 30 MiB free, Windows started paging GPU memory, and a request could hang at "reading the prompt" with the GPU at "100%" but low power. The head now loads first. The same 700 MiB reserve really stays free (about 500 MiB with everything loaded), and decode was not slower in our tests (Q2_0 at 32K: 78-79 tok/s vs 71.7 before). If free VRAM still ends below 256 MiB, the start-up line now says which --vram-reserve-mib to add.
• Use it from your phone or another PC (#26, thanks @XeonG). START-HERE.bat --setup --host 0.0.0.0 --api-key <secret> (or "host": "0.0.0.0" and "api_key" in strata-<model>.json). The server window prints this PC's network address, warns if there is no API key, and on Windows shows how to allow it through the firewall.
• --fit-max-tokens (#24, thanks @pjgmobile). Optional: a request whose max_tokens would run past the context gets a shorter answer instead of a 400 error (for agents that always ask for their full output cap). Off by default; turn it on with "fit_max_tokens": true in strata-<model>.json. A prompt that already fills the context is still refused.
• Experimental speed projection (EXPERIMENTAL, off by default). A small control vector that setup can turn on for the original Qwen3.8-Flash-Next (it asks; the answer defaults to no). It changes how the model answers: the vector's own package describes it as a refusal-direction projection, so the model declines far fewer requests. It is not an engine speed-up: on the same text it costs 0.2-0.4% per token, and a chat's speed with it depends on what the model writes. When it is on, the web app's Sampling drawer (or "experimental_speed_projection": false in a request) switches it off per chat. Read data/experimental-speed-projection/README.md and the section in docs/DETAILS.md before turning it on.

Updating: get the latest files (git pull or download the repo again), then run START-HERE.bat with the model window closed. It updates the engine to 0.1.9 by itself. To use the new options, run START-HERE.bat --setup. On Linux, ./setup.sh builds the engine from source as before.

The ready-made Strata engine for Windows (RTX 30 / 40 / 50: sm_86, sm_89, sm_120 + PTX), CUDA 13.0.

You don't need to download this yourself: START-HERE.bat fetches it (and NVIDIA's cuBLAS from pip), so no compiler or CUDA Toolkit is needed, only an NVIDIA driver 580 or newer.

Contents: strata.exe (the engine), strata-vision.exe (the optional image encoder), BUILD.json.