What's Changed
🚀 Features
- Integrate DeepEPv2 by @irexyc in #4783
- feat(pytorch): support Kimi K2.6 by @qescccczmr in #4846
- feat(kv_connector): support mooncake store by @caikun-pjlab in #4903
💥 Improvements
- feat(chat-completions): server-side fan-out for n>1 choices by @lvhan028 in #4841
- perf: further optimize GLM-5.2 serving by @CUHKSZzxy in #4853
- perf(cuda): use PDL for paged attention and V4 prefill by @grimoire in #4861
- [ascend] update attn op_backend by @wanfengcxz in #4900
- perf(pytorch): optimize compact blocked FP8 MoE and route preparation by @grimoire in #4857
- perf(pytorch): reduce speculative decoding pre/post-processing overhead by @grimoire in #4877
- support page size that are not power of two by @irexyc in #4854
- feat: support structural_tag response_format for turbomind and pytorch engines by @windreamer in #4906
🐞 Bug fixes
- fix(turbomind): restore FP8 weight-only fallback on pre-sm90 GPUs by @lvhan028 in #4871
- fix(vl): raise a clear error on malformed data URLs by @SuperMarioYL in #4837
- fix(api): fix reponses interface by @caikun-pjlab in #4893
- Fix/dsv4 native transformers warmup by @grimoire in #4878
- fix: support inline system messages in Anthropic API by @lvhan028 in #4882
- fix: bound DSA prefill score memory by @CUHKSZzxy in #4896
- fix(build): correct GEMM kernel archive link order by @lvhan028 in #4910
- fix: reject unavailable GLM tool calls by @CUHKSZzxy in #4901
- fix(pytorch): avoid Triton miscompile in paged attention reduction by @lvhan028 in #4920
- fix Intern-S2-Preview-FP8 convert by @irexyc in #4923
🌐 Other
- [Fix] Validate cross-file Markdown link targets by @JimmyWang0417 in #4868
- Update README to Reflect EuroSys 2027 Paper Acceptance by @Youhe-Jiang in #4891
- docs: fix grammar in README by @MarkHe1222 in #4866
- improve(autotest): trim unused model configs and gate routed_experts on yaml by @littlegy in #4885
- build: remove flashinfer from CUDA runtime requirements by @CUHKSZzxy in #4902
- [ci] add base api eval test workflow by @zhulinJulia24 in #4874
- bump version to v0.17.0 by @lvhan028 in #4914
New Contributors
- @JimmyWang0417 made their first contribution in #4868
- @Youhe-Jiang made their first contribution in #4891
- @MarkHe1222 made their first contribution in #4866
- @qescccczmr made their first contribution in #4846
Full Changelog: v0.16.0...v0.17.0