What's Changed
- Fix CUDA version extraction in install.sh by @zaqxsw-dev in #2287
- feat(metal): improve runtime and aot compilation system by @EricLBuehler in #2288
- feat(release): split cuda workers for docker release by @EricLBuehler in #2289
- feat(install): cleanup install system for updating by @EricLBuehler in #2290
- feat(distributed): more sophisticated TP fallback and resolution system by @EricLBuehler in #2291
- chore(mistralrs-core): cap tojson indent and remove chat template panic paths by @eharris128 in #2286
- feat(quant): improve uqff resolution per model ids by @EricLBuehler in #2293
- Fix agentic loop not starting for registered tool callbacks by @aeiklorvy in #2279
- feat(models): Support LFM 2.5 (dense + MoE) and LFM 2.5 VL! by @EricLBuehler in #2295
- feat(cuda): preload cuda ptx modules by @EricLBuehler in #2296
- refactor(core): move code exec shims from lib.rs to proper location by @EricLBuehler in #2297
- docs(models): improve supported models readability by @EricLBuehler in #2298
- feat(multi-model): support planning for loading multi-model by @EricLBuehler in #2299
- feat(cuda): update cutlass git revision by @EricLBuehler in #2300
- feat(cpu): aarch64 cpu repacking kernels and performance improvements by @EricLBuehler in #2304
- feat(cpu): optimize decode and prefill performance on x86 and aarch64 by @EricLBuehler in #2311
- fix(tune): detect CUDA backend on Windows by @matt-minev in #2312
- fix(server): keep Responses output item IDs stable by @matt-minev in #2309
New Contributors
- @zaqxsw-dev made their first contribution in #2287
- @eharris128 made their first contribution in #2286
- @aeiklorvy made their first contribution in #2279
- @matt-minev made their first contribution in #2312
Full Changelog: v0.8.23...v0.9.0