What's Changed
- feat(core): concurrency fixes and scheduler improvements by @EricLBuehler in #2389
- perf: first-class FP8 and concurrent DFlash decode by @EricLBuehler in #2390
- feat(metrics): add mistralrs_sequences_capacity gauge by @EricLBuehler in #2391
- perf(core): optimize exact stochastic DFlash serving by @EricLBuehler in #2392
- feat(server): align server error codes to openai standard by @EricLBuehler in #2393
- feat(logging): expose separate prefill and decode t/s logging by @EricLBuehler in #2395
- fix(multimodal): keep Qwen media work out of decode by @EricLBuehler in #2396
- fix(ci): green the build for rust 1.98 and metal by @EricLBuehler in #2397
- feat(core): report cached prompt tokens in usage by @EricLBuehler in #2398
- perf(core): improve hybrid FP8 serving by @EricLBuehler in #2399
- feat(server): improve Anthropic Messages and Claude Code support by @matt-minev in #2400
- docs: refresh generated reference artifacts by @matt-minev in #2401
- fix(website): fix handling for clipboard by @EricLBuehler in #2403
- perf(cuda): scale FP8 and DFlash2 serving through C128 by @EricLBuehler in #2404
- feat(website): add details about v0.9.3 release by @EricLBuehler in #2405
- chore(cuda): clippy by @EricLBuehler in #2406
- feat(release): improve reliability, add caching by @EricLBuehler in #2408
- fix(core): align tool chat template JSON rendering by @atirna in #2371
- fix(lfm2): support legacy full_attn_idxs config field by @tugot17 in #2305
- fix(webui): send chat images as data URLs (fix HTTP 500 on image send) by @subin9 in #2338
- feat(core): bump cutile to v0.3.0 by @EricLBuehler in #2410
- feat(core): fp8 tensor-core gemv/gemm, cutile gdn prefill by @EricLBuehler in #2412
- feat(cutile): add fp8 fused moe and a load-time kernel autotuner by @EricLBuehler in #2413
- fix(cuda): restore GB10 builds with cudaforge 0.1.6 by @sizzlecar in #2415
- feat(fp8): support ModelOpt and compressed-tensors checkpoints with cutile kernels by @EricLBuehler in #2416
- feat(quant): add NVFP4 checkpoint loading and cuTile Blackwell acceleration by @EricLBuehler in #2418
New Contributors
- @atirna made their first contribution in #2371
- @tugot17 made their first contribution in #2305
- @subin9 made their first contribution in #2338
- @sizzlecar made their first contribution in #2415
Full Changelog: v0.9.2...v0.9.3