github vllm-project/vllm-omni v0.34.0

56 minutes ago

Highlights

This release includes 776 merged pull requests from 185 contributors, including 61 first-time contributors. vLLM-Omni v0.34.0 rebases onto vLLM 0.34.0 and moves autoregressive and generation stages to Model Runner V2. It adds MiniMax-H3-Image, FLUX.3 (Klein), PAN2, Hy-Nomas-Diffusers, Haru-4B, and Ming-3.0 speech models, alongside distributed duplex serving and realtime video streaming.

Key Improvements

  • Scale duplex serving across nodes. Deploy stage replicas across independent runtimes while keeping each session pinned to its replica group. External orchestrators can also run stages independently and select replicas through the experimental /v1/run API. (#10422, #6883, #9729)
  • Stream video during request execution. CausVid can deliver decoded chunks while generation is in progress. Noisy Pipeline Parallelism can batch compatible work across requests while preserving each session’s KV state. (#9926, #10049, #10122)

Model Support

  • Image generation and editing. Added MiniMax-H3-Image for text-to-image generation and multi-reference editing through the Images API, and FLUX.3 (Klein) with a native BF16 pipeline. (#10120, #9523)
  • Video generation. Added PAN2 for text-to-video and image-to-video, and Hy-Nomas-Diffusers for camera-controlled autoregressive video streaming. Kandinsky 6 gains Lite and Pro Distil checkpoints with integrated HD and Full HD super-resolution. (#8602, #9338, #9027)
  • Speech and audio. Added Haru-4B for full-duplex spoken dialogue, Ming-3.0-TTS Tiny for streaming speech and voice cloning, Ming-3.0 Audio Flash for speech recognition and streamed spoken responses, and Stable Audio 3 Medium for stereo text-to-audio generation. (#10301, #9119, #9130, #3973)
  • Segmentation and depth. Added SAM 3 and SAM 3.1 for promptable image segmentation, SAM 3 for video segmentation and tracking, and Video Depth Anything for video depth estimation. (#9543, #9579, #9714)
  • Robotics and driving. Added NVIDIA Alpamayo 2 Super and native Runway Praxis-1 policies for DROID and YAM. Restored π0.5 support and enabled multi-stage policy serving over OpenPI. (#9133, #10197, #10204, #9080, #9120)

Core Architecture & Runtime

  • Reduced Omni prefix-cache overhead under concurrent requests and added LMCache CPU offloading for autoregressive KV cache and the hidden states downstream speech stages need. (#9162, #9193, #1330)
  • Added CUDA IPC weight sharing so compatible, colocated HunyuanImage3 AR and DiT stages can share a single copy of the backbone weights. (#9333, #9761, #9850)

Audio, Speech & Realtime Serving

  • Expanded duplex tool-call controls, added semantic voice activity detection to turn-based Realtime, and improved /v1/realtime protocol compatibility. (#9909, #9170, #9265)
  • Added experimental Mooncake KV and hidden-state reuse for supported Qwen3-Omni turn-based and duplex deployments. Enabled multi-frame CUDA graph replay for the MiniCPM-o 4.5 Talker. (#10107, #10321, #10288)

Diffusion, Image & Video Generation

  • Expanded MiniMax-H3 with more FastH3 checkpoints, ControlNet Union serving, and video continuation from cached latents across requests. (#9167, #7479, #9342)
  • Added experimental Physis Same-Step pipelines for Wan, LTX, and MiniMax-H3. Supported diffusion pipelines can also run both classifier-free guidance branches in one denoiser call on a single rank. (#9154, #9797)

Quantization & Memory Efficiency

  • Added ConvRot INT8 and INT4 quantization for diffusion transformers on CUDA, and support for the MiniMax-H3 AutoRound checkpoint with an MXFP8 DiT and MXFP4 text encoder. (#9343, #9363)

Serving, Frontend & API Behavior

  • Added OpenTelemetry inference tracing and opt-in AudioSeal watermarking for supported audio generation. (#8836, #7366)

Platforms & Hardware Coverage

  • Updated the Ascend stack to vLLM-Ascend 0.34.0, added Model Runner V2 deployment profiles for supported ROCm workloads, and enabled Intel XPU sleep mode and diffusion graph capture. (#10773, #10819, #10820, #5485, #9306)

Breaking Changes

  • Model Runner V2 is required. Deployments that resolve any stage to model_runner: v1 fail at startup. Review custom deployment files and platform overrides before upgrading. Affected profiles include full Drama 3 and Fish-Omni speech serving, PersonaPlex duplex, VibeVoice, LFM3-Omni, and the Qwen3-TTS and Qwen3-Omni MUSA overrides. (#10365, Deployment compatibility)
  • Removed first-party support for JoyAI-Image-Edit and dropped the --shm-threshold-bytes CLI option. (#10393, #9992)

What's Changed

New Contributors

Full Changelog: v0.32.0...v0.34.0

Don't miss a new vllm-omni release

NewReleases is sending notifications on new releases.