📦 What’s New
⚡ Prefill Speed Upgrade (Qwen & Gemma Families)
A new attention engine dramatically accelerates prefill, with larger gains at longer context lengths (especially 16K+).
- 🚀 Up to 3.8× faster prefill
(qwen3:0.6b with a 32K-token prompt)
📈 Prefill Speed @ 32K Prompt (tokens/sec)
| Model | Before → After | Speedup |
|---|---|---|
| gemma3:1b | 1596 → 1755 | 1.1× |
| gemma3:4b | 673 → 926 | 1.4× |
| medgemma:4b | 673 → 926 | 1.4× |
| qwen3:0.6b | 236 → 1496 | 3.8× |
| qwen3:1.7b | 225 → 768 | 3.4× |
| qwen3:4b | 164 → 303 | 1.9× |
| qwen3-it:4b | 164 → 303 | 1.9× |
| qwen3-tk:4b | 164 → 303 | 1.9× |
| qwen3vl-it:4b | 164 → 303 | 1.9× |
| qwen3:8b | 150 → 260 | 1.7× |
| deepseek-r1-0528:8b | 150 → 260 | 1.7× |
🖼️ Note: For qwen3vl-it:4b, image understanding is also faster in this release.
🔗 Benchmark Results
- 📊 Gemma3 performance: https://fastflowlm.com/docs/benchmarks/gemma3_results/
- 📈 Qwen3 performance: https://fastflowlm.com/docs/benchmarks/qwen3_results/
🛠️ Vision Tool Calling (New)
- ✅ Tool calling is now supported on qwen3vl-it
- 🔍 Enables vision tool calling workflows
- 🎥 Demo: https://youtu.be/Rf6r0Fm1UVs?si=u45hBgFXyDeEKXxh
🔧 Client Compatibility Improvements
- Non-stream mode logic adjusted
- Improves compatibility with client applications