Speculative expert prefetch
Expert prefetch can speed up generation on devices with less RAM.
- Enable Expert prefetch in the Mac app, then reload the model. The setting is saved.
- CLI and server users can enable it with
--early-expert-read router. - Improved reader cleanup during cancellation, errors, and model reuse.
- Updated benchmarks and experiment documentation with M2 and M5 Pro results.
Thanks to @maxslamdunk for the expert-prefetch implementation in #187.