- Add DFlash draft model support
- Add LoRA support
- Add n-gram (suffix) drafting
- Add AfmoeForCausalLM
- Add RYS layer map (see here)
Trace-driven SD benchmark (Qwen3.6-27B-EXL3 4.15 bpw, RTX 6000 Pro)
| Category | Baseline | N-gram/suffix | DFlash |
|---|---|---|---|
| Agentic, code | 55.98 t/s | 89.58 t/s, 1.60x | 140.61 t/s, 2.51x |
| Agentic, curl | 54.03 t/s | 74.62 t/s, 1.38x | 125.94 t/s, 2.33x |
| Coding | 59.21 t/s | 75.34 t/s, 1.27x | 177.67 t/s, 3.00x |
| Creative | 59.10 t/s | 67.26 t/s, 1.13x | 89.19 t/s, 1.50x |
| Creative (reasoning) | 59.03 t/s | 64.25 t/s, 1.09x | 93.54 t/s, 1.58x |
| Translation | 58.11 t/s | 55.39 t/s, 0.95x | 75.73 t/s, 1.30x |
| Translation (reasoning) | 58.08 t/s | 80.21 t/s, 1.38x | 119.43 t/s, 2.06x |
Full Changelog: v0.0.30...v0.0.31