What's Changed
🚀 Features
- Support Interns2 mobius by @RunningLeon in #4816
- feat: support GLM-5.2 by @CUHKSZzxy in #4737
- Intern-S2-Mobius meta-MoE support, MoE gate v2, CP attention fixes by @lzhangzz in #4835
- Add TurboMind ViT support for InternVL and Qwen VL models by @irexyc in #4719
- feat: add Hy3 support, MTP, and FP8 optimizations by @yidingcheng0206 in #4815
💥 Improvements
- refactor: report cache usage directly by @lvhan028 in #4798
- SM90 native BF16/FP8 GEMM kernels, fused-SiLU quantization, and linear test harness by @lzhangzz in #4795
- refactor: split api server endpoints by @lvhan028 in #4797
- refactor(pytorch): derive CUDA step metadata from selected operators by @grimoire in #4805
- optimize and modularize SSM prefix caching by @grimoire in #4788
- perf(guided-decoding): optimize with async D2H copy and xgrammar v0.2.1 by @windreamer in #4605
- refactor(serve): split chat_completions endpoint into a package by @lvhan028 in #4840
- feat(chat-completions): add usage.completion_tokens_details by @lvhan028 in #4842
- feat(pytorch): add optimized Gluon blocked FP8 GEMM for Hopper by @grimoire in #4830
- perf(pytorch): add opt-in torch.compile for decode CUDA graphs by @grimoire in #4808
- Ssm prefix cache non aligned by @grimoire in #4799
- perf: optimize GLM-5.2 serving by @CUHKSZzxy in #4827
- refactor: separate request preprocessing from generation by @lvhan028 in #4856
🐞 Bug fixes
- [Bugfix] Fix PyTorch H2D input lifetime across CUDA streams by @grimoire in #4792
- fix(serve): reject empty/falsy prompt input in format_prompts and AsyncEngine.generate by @SuperMarioYL in #4803
- Fix ray mp duplicate output by @RunningLeon in #4833
- fix(turbomind): dispatch cuMemcpyBatchAsync by CUDA runtime version by @lvhan028 in #4838
- fix(disagg): use JSON instead of pickle for P2P ZMQ requests (#4804) by @Anai-Guo in #4812
- fix(serve): emit signatures for Anthropic thinking blocks by @matrix72c in #4851
- Fix int4 KV cache quantization range when the packed head width is not a power of two by @truong-v in #4850
- fix: harden serving request validation by @lvhan028 in #4872
- fix: fix allgather/allgather2d for cuda-ipc when byte_width is not multiple of uint by @irexyc in #4873
📚 Documentations
- docs: update recent model support by @CUHKSZzxy in #4855
- docs,tests: cover Qwen3.8 preserve_thinking support by @lvhan028 in #4869
🌐 Other
- docs: remove non-existent
--enable-metricsflag from metrics/spec_decoding guides by @latent-9 in #4809 - Upgrade to cu130 by @RunningLeon in #4753
- TEST: update turbomind qwen3.5 config by @littlegy in #4778
- chore: use python3.12 for docformatter pre-commit hook by @lvhan028 in #4839
- ci: support CUDA 13.0 Docker builds and publishing by @lvhan028 in #4817
- TEST: update deepseekv4-flash config by @littlegy in #4836
- [ci] Adjust evaluation gate benchmark datasets to reduce runtime and extend coverage by @zhulinJulia24 in #4834
- [ci] remove old models and refactor interface testcase by @zhulinJulia24 in #4806
- bump version to v0.16.0 by @lvhan028 in #4847
New Contributors
- @latent-9 made their first contribution in #4809
- @SuperMarioYL made their first contribution in #4803
- @yidingcheng0206 made their first contribution in #4815
- @matrix72c made their first contribution in #4851
- @truong-v made their first contribution in #4850
Full Changelog: v0.15.0...v0.16.0