--bench now warms up before it measures prefill (reading the prompt), so speed reports compare fairly.
Update: git pull, then ./setup.sh (Windows: START-MAYA.bat). This release changes only the installer script, so nothing recompiles.
What's new
--bench: one unmeasured prompt is read before the 2k and 8k prefill measurements. The 2k figure used to be the first prompt after start-up, with the caches still cold, and could read lower than the 8k one (152 vs 238 tokens/s on a Titan RTX pair). Reported by @dummerjindabin (#8).--benchand--report: a request that only read a prompt shows just its prefill time in the engine lines, not a meaningless decode (writing the answer) speed (it read "96,000 tokens/s").