Niko1221/Strata v0.1.3

Strata v0.1.3

on GitHub

A conversation cache, no more CPU load while idle, and a Linux start-up fix. Thanks to @Mirtraxxx, @Vistawizard and @coolio986 for the pull requests.

• Follow-up turns no longer re-read the whole conversation (#8, @Mirtraxxx).
  • The engine keeps the chat between requests, plus up to 6 checkpoints (~118 MB of RAM each), and reads only what is new.
  • On an RTX 5070, a follow-up in a 7.7K-token chat now starts in 0.3 s instead of 12.9 s.
  • Pictures are matched by their content, so a different picture never reuses another one's cache.
  • --prompt-cache 0 turns it off.
• Short messages start faster (#10, @Mirtraxxx). Prompt parts of up to 64 tokens go through the decode path, so "hi" is answered in about 0.2 s. --short-read 0 turns it off.
• No CPU load between requests (#9, @Mirtraxxx, fixes #4). The expert-pool workers now sleep when idle instead of spinning: 5 busy cores → 0 on a Ryzen 5 7600, with the same speed while generating.
• Linux: the engine no longer hangs at start (#14, @Vistawizard, fixes #13 and #11). A missing pair of braces made the CPU pool start thousands of workers. The --pool-workers workaround is no longer needed.
• No more empty replies when a client sends two requests at once (#7, @coolio986), for example opencode's title request next to the main turn. max_tokens: -1 now means "the default" instead of zero tokens.

Engine protocol: DONE gains three fields (drafts accepted, drafts offered, prompt tokens reused), and the engine now prints RESUME, PP and REUSED lines. The server included here reads them.

Updating: get the latest files (git pull or download the repo again), then run START-HERE.bat with the model window closed. It sees the older engine and downloads this one before starting the model. From now on START-HERE.bat checks the engine every time it starts, so later releases only need the new files. If the download can't happen (no internet, model still running), it starts your installed engine and tries again next time. On Linux, ./setup.sh builds the engine from source as before.

The ready-made Strata engine for Windows (RTX 30 / 40 / 50: sm_86, sm_89, sm_120 + PTX), CUDA 13.0.

You do not need to download this yourself: START-HERE.bat fetches it (and NVIDIA's cuBLAS from pip), so no compiler or CUDA Toolkit is needed - only an NVIDIA driver 580 or newer.

Contents: strata.exe (the engine), strata-vision.exe (the optional image encoder), BUILD.json.