Longer contexts and faster full-attention decoding for Gemma 4.
- Add context sizes up to 256K, with memory checks that account for the selected expert cache and migration of incompatible saved settings.
- Use grouped full attention for Gemma 4 while preserving the previous kernel's reduction and accumulation order.
- Report separate prefill and decode timings in server logs.
- Limit the Mac app to 32 image attachments per message; the remaining context may fit fewer.
The app version is 0.9.0. Context, sampling, and expert-cache defaults remain unchanged.
One M5 Pro comparison after a 110,000-token prompt improved generation from 8.14 to 18.02 tokens/s, with all 256 output token IDs matching. Both runs used 32 expert-cache slots. See the measurement and its limits.
Long-context retrieval quality and constrained-memory validation remain incomplete. PR #182 records the accepted follow-ups for rejected reloads, context suggestions, and server/CLI image limits.
This is a source-only release.
Changes: #182
Full changelog: 0.8.1...0.9.0