vllm-project/vllm-omni v0.32.0

on GitHub

Highlights

vLLM-Omni v0.32.0 aligns with vLLM 0.32.0 and brings multi-node execution, expanded realtime serving, new image and audio models, and release-branch stability fixes since v0.30.0.

Runtime and realtime serving

• Add optional Ray-based multi-node inference, session-aware replica placement, and multi-frontend full-duplex serving. (#7938, #8913)
• Add server VAD for turn-based /v1/realtime and RL weight-transfer support. (#8600, #7686)
• Improve request failure isolation, repeated-stage-failure handling, and FlashInfer compatibility with torch.compile(fullgraph=True). (#9470, #9275, #9397)

Models and generation

• Add YuE2-3B text-to-music, Qwen-Image-2.1 / 2.1-Turbo, Kandinsky6, and MOSS-Image. (#7886, #8099, #8699, #8537, #8926)
• Add TaoMate-H3 realtime AR-diffusion streaming and an LTX 2.5 single-stage text-to-audio pipeline. (#8268, #6489)
• Add optional single-stage Qwen3-TTS with in-Talker streaming codec decoding; improve PersonaPlex batched execution and CUDA graphs. (#8259, #8441)
• Add CUDA FA3 FP8 diffusion attention and online FP8 quantization for MammothModa2 DiT. (#8965, #7221)

Compatibility

• Requires Python >=3.11,<3.14 and the matching vLLM 0.32.0 release. (#9066)
• Output-type names require canonical spellings: text, image, audio, latent, and token_ids. Legacy aliases and malformed values are rejected. (#7843)

Release stabilization

This release includes curated fixes on release/v0.32.0, including the idle MoE LoRA-context repair for HunyuanImage-3. (#9515)

Source commit: 5917730a4b02aedb44d374173cdc1b10484ccd52.

Release artifacts

• PyPI: vllm-omni 0.32.0.
• CUDA Docker: vllm/vllm-omni:v0.32.0, with linux/amd64 and linux/arm64 images. The latest tag points to this release.
• Release downloads: wheel, source distribution, and SHA256SUMS.

Install the matching CUDA 13.0 packages in a fresh environment:

uv venv --python 3.12
source .venv/bin/activate
uv pip install "vllm==0.32.0" "vllm-omni==0.32.0" --torch-backend=cu130

Or pull the CUDA container:

docker pull vllm/vllm-omni:v0.32.0

Full changelog: v0.30.0...v0.32.0