InternLM/lmdeploy v0.15.0

on GitHub

What's Changed

🚀 Features

• Support long-context and MTP prefix-cache hits by @grimoire in #4688
• [Feature] Add guided decoding support for speculative decoding by @windreamer in #4559
• feat(turbomind): memory allocator, object cache, and scheduler integration by @lzhangzz in #4717
• feat: add AgRs all2all backend by @irexyc in #4739
• DeepSeek V4 support by @grimoire in #4554
• Support memdecode by @lvhan028 in #4767

💥 Improvements

• Force blksize=128 for linear attention on ascend by @jinminxi104 in #4705
• refactor: unify interleaved MRoPE rotary embedding by @CUHKSZzxy in #4644
• Refine multi-node support on ascend-A3 by @jinminxi104 in #4711
• [Improve]: Remove dlblas from lmdeploy by @RunningLeon in #4682
• replace sync with wait event in h2d by @grimoire in #4709
• Respect --server-port in DP mode when proxy-url is set by @lvhan028 in #4712
• add --language-model-only for text-only VLM inference and remove --disable-vision-encoder by @lvhan028 in #4716
• Optimize TTFT by @grimoire in #4695
• Optimize BaseResponseParser streaming and add parser benchmark by @lvhan028 in #4697
• Support fp8 moe only for qwen3.5 by @RunningLeon in #4740
• clear runtime state in sleep by @grimoire in #4729
• feat(serve): add --generation-config CLI for server sampling defaults by @lvhan028 in #4708
• fix: gzip torch profiler traces by default by @CUHKSZzxy in #4747
• Optimize tp fp8 moe for small average router per expert by @grimoire in #4751
• refactor(pytorch): clarify scheduler and input-maker control flow by @grimoire in #4727
• Guard DP dummy inputs around pending work by @grimoire in #4738
• Remove interactive chat and make inference stateless by @lvhan028 in #4730
• feat(turbomind): Derive composable TurboMind parallel configurations by @lzhangzz in #4769
• Add generic tensor copy and architecture-aware Gated Delta Rule support by @lzhangzz in #4757
• Add GDR CP controls and legacy kernel override by @lzhangzz in #4779
• fix(turbomind): fix zero-centered RMSNorm for Qwen3.5 by @lzhangzz in #4790

🐞 Bug fixes

• fix prefix caching by @grimoire in #4700
• fix _reduce_split_kernel for triton 3.5.1 by @irexyc in #4696
• fix triton fp8 all_reduce group by @grimoire in #4702
• fix(serve): use unique chatcmpl id for chat completions responses by @lvhan028 in #4707
• [Bugfix] Fix ImportError in get_chat_template for builtin chat-template names by @waynehacking8 in #4690
• [Bugfix] Fix InternVL/InternVL3 LoRA loading TypeError in adapter fallback by @waynehacking8 in #4684
• Force blksize=128 when head_dim=256 on ascend by @wanfengcxz in #4723
• fix HCCL port conflict on multi-dp rank startup(single node) by @wanfengcxz in #4722
• fix: fail fast on invalid serve parsers by @CUHKSZzxy in #4701
• fix: parse multimodal tool messages by @CUHKSZzxy in #4680
• Reprobe once for health request by @RunningLeon in #4703
• fix: enable graph capture during DP warmup decoding by @wanfengcxz in #4728
• fix: proper XGrammar integration for guided decoding by @windreamer in #4726
• fix: release multimodal payload after mp handoff by @CUHKSZzxy in #4725
• Fix MTP recurrent state round-trip for Qwen3.5 by @RunningLeon in #4744
• Fix mtp dpmeta and guard the warmup by @RunningLeon in #4741
• Fix GLM MTP by @RunningLeon in #4749
• fix false negative healthy status check for turbomind by @irexyc in #4745
• fix(vl): forward tools to multimodal chat templates by @lvhan028 in #4759
• fix: restrict remote media domains by @CUHKSZzxy in #4734
• fix(turbomind): restore INT8 KV quant-param offset in block layout by @lzhangzz in #4764
• fix(turbomind): avoid async TP shutdown deadlock by @lzhangzz in #4770
• fix: bound grouped GEMM scheduling by routed tokens by @lzhangzz in #4771
• feat(turbomind): enable SM90 GDR PDL and fix CP matrix layout by @lzhangzz in #4787
• fix(serve): auto-generate batch completion sessions to avoid id collision (#4773) by @Anai-Guo in #4774
• feat(turbomind): restore metrics reporting by @lvhan028 in #4768
• fix: skip interns2 preview time-series weights by @CUHKSZzxy in #4801
• fix(serve): handle missing logprobs attrs on TurbomindEngineConfig in Anthropic messages by @lvhan028 in #4800

🌐 Other

• bump version to v0.14.0 by @lvhan028 in #4689
• TEST: Improve tool test by @littlegy in #4632
• chore: update deepgemm revision by @CUHKSZzxy in #4713
• TEST：Add H Prefix Cache Test, HF Path Conversion, and Ascend Multi-node Startup Config by @littlegy in #4706
• chore: remove deprecated model support by @CUHKSZzxy in #4693
• [Ascend] support qwen35 mtp on Ascend-A3 by @wanfengcxz in #4721
• ci: pin jlumbroso/free-disk-space to a full commit SHA by @kobihikri in #4748
• fix: install xgrammar for jetson docker by @CUHKSZzxy in #4758
• update turbomind builder image by @irexyc in #4752
• fix(autotest): drop flaky text inequality in same session_id generate test by @littlegy in #4766
• fix(benchmark): seed NumPy RNG so --seed controls random-mode lengths (#4784) by @Anai-Guo in #4785
• update anthropic endpoint test by @littlegy in #4594
• Update multimodal toolcall tests by @littlegy in #4742
• Remove obsolete C++ tests by @lzhangzz in #4796
• [Docs] Fix typos in contribution guide by @cupkk in #4807
• bump version to v0.15.0 by @lvhan028 in #4791

New Contributors

• @kobihikri made their first contribution in #4748
• @Anai-Guo made their first contribution in #4785
• @cupkk made their first contribution in #4807

Full Changelog: v0.14.0...v0.15.0