What's new in 4.2.0 (2026-11-07)

These are the changes in inference v4.2.0.

New features

• feat: support service relaunch on Windows by @qinxuye in #5868
• feat: check and install Xinference updates from the Web UI by @qinxuye in #5873
• FEAT: Add optional Worker controller-session admission by @qinxuye in #5901
• feat: add Mooncake Store storage to Xavier LMCache by @qinxuye in #5905
• feat: support llama.cpp engine for DeepSeek-V4.1-Flash-1017 by @llyycchhee in #5912
• feat(ocr): add Unlimited-OCR-2 support by @Minamiyama in #5913
• feat(ocr): add OvisOCR3 support by @Minamiyama in #5914
• feat(ocr): add MiniCPM-OCR-1.0 support by @Minamiyama in #5916
• feat: add Qwen NVFP4/YaRN support and update Gemma-4 GGUF runtime by @qinxuye in #5919
• FEAT: Add optional Supervisor control extension mechanisms by @qinxuye in #5911
• FEAT: Share Xavier prefix caches across engines by @qinxuye in #5917
• feat: automatically retain GPU weights on supported model launches by @qinxuye in #5924
• FEAT: Extend Qwen automatic YaRN engine support by @qinxuye in #5925
• feat(video): add MiniMax-H3 to vLLM and SGLang with Lightning by @qinxuye in #5930
• FEAT: Add pluggable control calls and Supervisor journal storage by @qinxuye in #5931
• feat: support multi-replica weight retention and rolling reload by @qinxuye in #5944
• feat: add adaptive vLLM multimodal preflight by @m199369309 in #5947
• feat(ui): group navigation extensions and complete catalog translations by @qinxuye in #5975
• feat: show featured model badges and refresh recommendations by @qinxuye in #5986
• feat: support Qwen3 realtime audio with vLLM by @qinxuye in #5993
• feat: support point virtual env index url by @llyycchhee in #5995
• feat(llm): expose native reasoning controls in APIs and Web UI by @qinxuye in #6001
• feat: support offline compressed-tensors checkpoints with vLLM by @qinxuye in #6000
• feat: share hybrid prefixes across Xavier vLLM replicas by @qinxuye in #6012
• feat: add session affinity for chat replicas by @qinxuye in #6013
• feat: prioritize agent KV prefixes across tool waits by @qinxuye in #6017
• feat: account for vLLM capacity in chat affinity by @qinxuye in #6016
• feat: extend Xavier agent cache lifecycle to tiered storage by @qinxuye in #6018
• feat(tts): support Audio8-TTS-Realistic-0.6b by @Minamiyama in #6003
• feat(llm): add MiniCPM-V-4.7 SGLang support by @Minamiyama in #6004
• feat(llm): support request priority and TRT-LLM 1.3 P/D by @qinxuye in #6019
• feat(llm): expose loaded runtime context length by @qinxuye in #6023
• feat(webui): configure worker and GPU placement per added replica by @m199369309 in #6021
• feat(webui): show elapsed time while models are launching by @m199369309 in #6022

Enhancements

• ENH: Add model-local realtime session runtime and gateway by @m199369309 in #5850
• ENH: Add opt-in Fun-ASR-Nano segmental realtime transcription by @m199369309 in #5851
• ENH: Add optional Nano realtime policies diagnostics and launch options by @m199369309 in #5852
• ENH: Add built-in model performance tests to the Web UI by @qinxuye in #5904
• ENH: Add LMCache history to Xavier prefill across engines by @qinxuye in #5985
• perf(ci): use lightweight checks for documentation-only changes by @qinxuye in #5927

Bug fixes

• fix(client): stop reading streams after the completion marker by @jackiectl2 in #5853
• fix(embedding): report each batched caller's token usage by @Jackmeson1 in #5857
• fix(audio): exclude broken preshed wheels from Kokoro dependencies by @qinxuye in #5855
• fix(flexible): handle omitted launcher arguments by @Jackmeson1 in #5859
• fix(llm): map native thinking modes in the shared chat renderer by @Jackmeson1 in #5861
• fix(llm): render text and image content in MiniCPM-V 2.6 and 4.5 templates by @Jackmeson1 in #5863
• fix(llm): normalize GLM-4.1V/4.5V text-only content blocks by @Jackmeson1 in #5867
• fix(llm): pass request chat template kwargs to the Qwen2-VL processor by @Jackmeson1 in #5865
• fix(flexible): preserve registrations during component handoff by @qinxuye in #5870
• fix(core): reject stale router assignment writes by @kartsan03 in #5869
• fix(api): answer 400 for invalid requests on non-LLM endpoints by @Jackmeson1 in #5875
• fix(sglang): send seed as sampling_seed by @David-Wu1119 in #5876
• fix(llama.cpp): pass repetition_penalty as repeat_penalty by @David-Wu1119 in #5877
• fix(llama.cpp): keep json_object response_format for chat by @David-Wu1119 in #5878
• fix(llm): restore the official Hy-MT2-7B and 30B-A3B chat templates by @David-Wu1119 in #5879
• fix(llm): restore the official GLM-4.5 chat template by @David-Wu1119 in #5880
• fix(llm): restore the official GLM-4.1V-Thinking chat template by @David-Wu1119 in #5881
• fix(llm): use Hy-MT2-7B's own eos tokens as stop tokens by @David-Wu1119 in #5882
• fix(llm): replace pinned Hugging Face revisions that no longer exist by @David-Wu1119 in #5883
• fix(embedding): report 768 dimensions for jina-embeddings-v2-base-en by @David-Wu1119 in #5884
• fix(cli): match size and quantization in xinference engine like launch by @MohammadHijjawi97 in #5886
• fix(rerank): drop the nonexistent Q3_K file from bge-reranker-v2-m3 GGUF by @David-Wu1119 in #5885
• fix(llm): keep every text part and one system message in the InternVL prompt by @Jackmeson1 in #5888
• fix(llm): accept the documented string audio_url in chat messages by @Jackmeson1 in #5890
• fix: keep model launches visible and clear failed records in the Web UI by @qinxuye in #5902
• fix(api): pass every timestamp_granularities[] value to audio models by @Jackmeson1 in #5892
• fix(llm): restore the official GLM-4.5V chat template by @David-Wu1119 in #5893
• fix(llm): restore the official Qwen2.5-Coder chat template by @David-Wu1119 in #5894
• fix(transformers): select the dedicated classes for deepseek-vl2 and glm-4v by @Jackmeson1 in #5896
• fix(image): accept a one-element prompt list in SenseNova-U1 image editing by @Jackmeson1 in #5898
• fix(llm): give Baichuan-M2 the Qwen tool parser by @Jackmeson1 in #5900
• fix(core): keep the working directory out of prepared-Python Worker relaunches by @kartsan03 in #5903
• fix(model): launch local model_uri models on Windows without symlink privilege by @jaeminbird in #5906
• fix(llm): accept forced tools that omit parameters by @jaeminbird in #5908
• fix(model): replace a dangling model_uri cache link for image, audio, video and world models by @NgoQuocViet2001 in #5915
• fix(ocr): install engine dependencies when --model-engine is omitted by @Caldalis in #5920
• fix(model): validate engine and lightning options before caching by @Jackmeson1 in #5922
• fix: preserve multiple choices in OpenAI-compatible endpoints by @mikemikimike in #5918
• fix: include user-defined LLM families in _resolve_architectures by @Ricardo-M-L in #5351
• fix(llm): make the deepseek-r1-distill-llama GGUF specs resolve to real files by @David-Wu1119 in #5928
• fix(llm): use the Mistral-Large-Instruct-2407 shard names for its split GGUF files by @David-Wu1119 in #5929
• fix(audio): describe ModelScope-only builtin registrations by @Jackmeson1 in #5935
• fix(core): canonicalize virtualenv package override names by @Jackmeson1 in #5937
• fix(ocr): validate the engine before caching by @Jackmeson1 in #5939
• fix(client): use error helpers in remaining request paths by @Jackmeson1 in #5941
• fix(embedding): select llama.cpp for matched GGUF specs by @Jackmeson1 in #5943
• fix: normalize multimodal model operation errors by @m199369309 in #5946
• fix(llm): match local paths across registered hubs by @Jackmeson1 in #5949
• fix(model): validate auxiliary GGUF selections before caching by @Jackmeson1 in #5951
• fix(client): normalize trailing slashes in endpoints by @Jackmeson1 in #5953
• fix(llm): use the official QwQ-32B-Preview chat template by @David-Wu1119 in #5957
• fix(llm): give the DeepSeek-R1/V3 GGUF quantizations with different shard names their own specs by @David-Wu1119 in #5954
• fix(llm): use the official HuatuoGPT-o1-Qwen2.5 chat template by @David-Wu1119 in #5958
• fix(llm): use the official Ornith chat templates by @David-Wu1119 in #5956
• fix(llm): detect more templates that iterate tool-call arguments as a mapping by @David-Wu1119 in #5960
• fix(llm): keep non-ASCII text in the chat template tojson filter by @David-Wu1119 in #5961
• fix(image): download SD 1.5 and SD inpainting from the stable-diffusion-v1-5 repos by @David-Wu1119 in #5963
• fix(audio): pin CosyVoice-300M Hugging Face revisions that exist by @David-Wu1119 in #5964
• fix(llm): use Meta's repo names for the llama-3.2-vision base models by @David-Wu1119 in #5966
• fix(llm): use the current official Qwen3-Coder chat template by @David-Wu1119 in #5959
• fix(llm): drop the extra "Qwen/" from the Qwen3-235B-A22B MLX repo id by @David-Wu1119 in #5967
• fix(llm): offer the gemma-3 MLX bf16 repos instead of missing fp16 ones by @David-Wu1119 in #5968
• fix(llm): set mistral-nemo-instruct's context length to 128K by @David-Wu1119 in #5969
• fix(llm): list only the Qwen3-8B and Qwen3-235B-A22B GGUF files that exist by @David-Wu1119 in #5970
• fix(llm): swap the deepseek-r1 and deepseek-r1-distill-qwen chat templates back by @David-Wu1119 in #5955
• fix: preserve matched Anthropic stop sequences by @mikemikimike in #5945
• fix(llm): drop deepseek-prover-v2's family stop id that is " for the 7B by @David-Wu1119 in #5962
• fix(llm): use the official DeepSeek-V3 chat template by @David-Wu1119 in #5965
• fix(llm): match the Qwen3-2507 GGUF specs to the unsloth file names by @David-Wu1119 in #5971
• fix(llm): give Hy-MT2-7B's HY-MT2-7B-named GGUF files their own spec by @David-Wu1119 in #5972
• fix(llm): give QwQ-32B's dot-named Q8_0 GGUF file its own spec by @David-Wu1119 in #5973
• fix(llm): point the GLM-4-32B-0414 GGUF spec at the 32B repo by @David-Wu1119 in #5974
• fix(auth): prevent expired-session IP bans and enable admin recovery by @qinxuye in #5994
• fix(llm): point qwen2.5-instruct 72B MLX 2bit at the 72B ModelScope repo by @David-Wu1119 in #5976
• fix(llm): match the Qwen3.8-27B GGUF quantizations to the current unsloth files by @David-Wu1119 in #5978
• fix(llm): parse Gemma tool calls whose property names are not plain ASCII words by @ZhuLinsen in #5999
• fix(llm): list the Qwen3.5-397B-A17B GGUF quantizations that exist by @David-Wu1119 in #5977
• fix(llm): detect templates that tojson tool-call arguments without a string check by @David-Wu1119 in #5979
• fix(llm): render qwen1.5/qwen2 tool-call history as valid <tool_call> blocks by @David-Wu1119 in #5980
• fix(llm): drop GGUF quantizations that have no file on either hub by @David-Wu1119 in #5981
• fix(llm): list only the DeepSeek GGUF quantizations that exist by @David-Wu1119 in #5982
• fix(llm): match the SeaLLM v2 and v2.5 GGUF specs to their file names by @David-Wu1119 in #5983
• fix(image): list Qwen-Image's BF16 GGUF file instead of a missing F16 by @David-Wu1119 in #5987
• fix(llm): add the shard lists for two GGUF quantizations uploaded as splits by @David-Wu1119 in #5988
• fix(llm): correct two Qwen2.5-Instruct GGUF shard lists by @David-Wu1119 in #5989
• fix(model): point ModelScope sources at repos that hold the files by @David-Wu1119 in #5991
• fix(llm): stop pointing MLX specs at repos that do not exist by @David-Wu1119 in #5992
• fix(llm): accept tool calls for qwen2.5-instruct-1m and QwQ-32B-Preview by @David-Wu1119 in #5996
• fix(llm): use Seed-OSS's own reasoning tags by @David-Wu1119 in #5997
• fix(llm): accept OpenAI tool messages and tool definitions in qwen1.5/qwen2 by @David-Wu1119 in #6002
• fix(llm): reject registrations with colliding cache paths by @Jackmeson1 in #6005
• fix(core): reject n_worker above the worker count before publishing the launch by @kartsan03 in #6007
• fix(client): accept image URLs in image_edit by @Jackmeson1 in #6008
• fix(client): avoid blocking the event loop when constructing AsyncClient by @Jackmeson1 in #6009
• fix(llm): parse Seed-OSS tool calls by @David-Wu1119 in #5998
• fix(llm): stop pointing specs at repos deleted from Hugging Face by @David-Wu1119 in #5990
• fix(llm): align the DeepSeek-V3.2 tool prompt with its parser by @Jackmeson1 in #6011
• fix(llm): validate recorded custom LLM cache sources by @Jackmeson1 in #6006
• fix(client): release streams closed before reading by @bingdongni in #6015
• fix: track replica scale-up until models are ready by @m199369309 in #6020
• fix(core): check every chat replica during affinity failover by @kartsan03 in #6024
• fix: avoid false LMCache misses during concurrent reads by @qinxuye in #6025
• fix(video): resize the last frame to the first frame's size by @lbarrosby in #6026
• fix(image): pad every image when padding_image_to_multiple is set by @lbarrosby in #6027
• fix: republish evicted Xavier LMCache pages by @qinxuye in #6028

Documentation

• DOC: Note v4.2.0 support for starting benchmark tests from the Web UI by @qinxuye in #5910
• DOC: fix dead LangChain, LlamaIndex, Dify and GitHub links by @sarankumar1325 in #5923
• DOC: Fix video guide translations and sidebar labels by @qinxuye in #5926
• doc: complete Anthropic multimodal translations by @qinxuye in #5984

Others

• docs: add the v4.1.0 row to the release notes index and locale catalogs by @martinma51 in #5871
• test(deploy): cover exited siblings alongside live process monitors by @qinxuye in #5872

New Contributors

• @jackiectl2 made their first contribution in #5853
• @jaeminbird made their first contribution in #5906
• @Caldalis made their first contribution in #5920
• @sarankumar1325 made their first contribution in #5923
• @ZhuLinsen made their first contribution in #5999
• @bingdongni made their first contribution in #6015
• @lbarrosby made their first contribution in #6026

Full Changelog: v4.1.0...v4.2.0