github vllm-project/vllm-omni v0.30.0
on GitHub

4 hours ago

Highlights

This release features 504 merged changes from 183 contributors, including 94 new contributors.

vLLM-Omni v0.30.0 is led by three major advances: 1) a unified full-duplex serving framework, 2) native cross-stage KV and multimodal payload transfer, and 3) interactive world-model and streaming video generation. The release rebases onto vLLM 0.30.0, expands model support across speech, image, audiovisual generation, and VLA, and improves batching, memory efficiency, and deployment scalability.

Key Improvements

  • Unified full-duplex serving around engine-owned sessions, separating duplex orchestration from turn-based execution and introducing shared model-plugin interfaces. MiniCPM-o 4.5 uses the new framework, with AURA adding multimodal interaction and multi-turn history. (#7413, #7633)
  • Extended scheduler-managed diffusion KV into cross-stage transfer and reuse, adding native Mooncake AR-to-DiT handoff, KV prefetch, and HunyuanImage3 cross-request paged prefix caching. Added NIXL transport for complete AR/encoder-to-diffusion conditioning payloads. (#7166, #7637, #7719, #6264)
  • Advanced interactive world-model and video execution, with LingBot World stepwise generation and mid-stream camera control, session-owned streaming VAE decode, and bounded worker-side video encoding and asynchronous chunk transfer. (#6844, #7198, #6533, #7048, #7406)

Core Architecture & Runtime

  • Rebased the project onto vLLM 0.30.0, including corresponding Ascend NPU integration updates. (#7820, #8027)
  • Introduced DuplexOmni, DuplexOmniEngine, and DuplexOrchestrator, moving session ownership and lifecycle management into the engine. Eligible models can serve duplex sessions and Chat Completions using the same engine and weights. (#7413)
  • Added parallel stage initialization with device-aware admission and locking, allowing independent stages and replicas to initialize concurrently. Centralized Omni configuration resolution and retained structured configuration through runtime startup. (#5224, #7292, #5140, #6849)
  • Added NIXL Omni connectors and complete conditioning-payload transfer, including text, visual/audio latents, and layout metadata for disaggregated MiniMax H3 execution. (#6093, #6264)
  • Added scheduler-level pause_generation(mode="keep") for request-level diffusion execution, preserving queued work and acknowledging device synchronization before returning. (#7685)

Model Support

  • Added MAGI-2 Preview with native text/image-to-video-and-audio generation, synchronized stereo audio, distributed execution, and Cache-DiT integration. (#5918)
  • Added Ming-Image-0.1-Design and Design-Layer, supporting text-to-image generation, single-reference image editing, and RGBA layer decomposition. (#8021)
  • Added Anima native text-to-image inference from local single-file checkpoints, and support for HunyuanImage-3.0-Instruct-Distil. (#4083, #4048)
  • Expanded SenseNova-U1.5-8B-MoT support with its distilled eight-step LoRA, and added the Boogu-Image Turbo native DMD execution path. (#6516, #6699)
  • Added Tencent AuK and AuK-Flash for instruction-driven speech generation and editing, Audio8 TTS Preview 0.6B, and Breeze-TTS-2 with a two-stage streaming AR/codec pipeline. (#7385, #6157, #7084)
  • Added an experimental JoyAI-VL-Interaction native multi-stage speech pipeline and an experimental Mage-VL duplex adapter and reference-backed offline integration. (#5352, #6537)
  • Added π0.5 (Pi0.5), extending vision-language-action model coverage. (#6950)

Audio, Speech & Realtime Serving

  • Moved the shared MiniCPM-o 4.5 and PersonaPlex duplex infrastructure out of the experimental namespace, and integrated AURA with the unified runtime for vision-only inputs, session history, and overlapping subsequent turns with draining speech output. (#6196, #7633)
  • Enabled event-driven orchestration by default for Qwen3-TTS and selected experimental Model Runner V2 in its default CUDA deploy profile. (#7088, #7781, #7930)
  • Added server-side VAD for turn-based omni models and a shared realtime web UI for MiniCPM-o and Qwen3-Omni. (#6618, #7585)
  • Improved TTS execution with OmniVoice variable-length attention and request batching, optimized MOSS-TTS Local batching and streaming codec execution, and bounded-window incremental CosyVoice3 vocoder streaming. (#6408, #7202, #7521)
  • Added Gepard-1.0 Speech API serving, configurable speech output sample rates, and Speech API streaming metrics. (#7499, #6553, #6853)

Diffusion, Image & Video Generation

  • Expanded MiniMax H3 with the full LightX2V Turbo LoRA matrix, direct FastH3 8-Step V2 checkpoint loading, latent-mask editing, and long-video latent continuation with driving audio. (#7062, #7610, #7465, #7838)
  • Extended LingBot World with Ulysses sequence parallelism, distributed streaming VAE decode, and experimental last-step KV reuse. (#6841, #7651, #7816)
  • Added BAGEL step execution and continuous batching, request-level batching for Boogu-Image, and migrated MammothModa2 DiT onto the shared diffusion runtime with request batching and Cache-DiT acceleration. (#6359, #6968, #7134, #7476, #7291)
  • Expanded SANA-Video 2B with tensor, CFG, and sequence parallelism, Cache-DiT, and CPU offload. Added CFG and sequence parallelism for Boogu-Image. (#5861, #5940, #5882, #6786, #5718)
  • Improved video output handling with chunked worker-side MP4 encoding and bounded asynchronous device-to-host transfers, reducing full-video materialization and producer-side blocking. (#7018, #7048, #7406)

Quantization & Memory Efficiency

  • Added component-selective offload policies and unified diffusion offload topology resolution. Improved HSDP startup through rank-0 shared weight loading and accelerated LoRA delta computation. (#5929, #7209, #7005)
  • Added Cosmos3 mixed W8A8/W8A16 and W4A4/W4A16 denoising, FP8 support for Boogu-Image and its MLLM, and online FP8 linears for LingBot World. (#6560, #6925, #7342, #7549)
  • Added MammothModa2 AR-stage FP8 KV cache, reduced MiniMax H3 VAE memory peaks, and introduced reusable exact projection caching with MiniMax H3 integration. (#7436, #7241, #7987)

Serving, Frontend & API Behavior

  • Added opt-in multi-process API serving through --api-server-count, allowing supported local EngineCore pipelines to share GPU stage engines across frontend processes. (#6923)
  • Added an initial RL rollout serving API with session creation, stepping, reset, close, and status operations for world_model_env integration. (#4770)
  • Expanded ComfyUI workflows with FastH3, MiniMax H3 reference inputs, latent editing, and MiniMax Music 3 generation. (#7456, #7483, #7898, #7516)
  • Extended benchmarking to image/video endpoints and duplex interaction, adding OmniInteract and Omni-DuplexEval performance metrics, accuracy evaluation, and detailed diffusion pipeline timings. (#4728, #7242, #7317, #6822)

Platforms & Hardware Coverage

  • Expanded Ascend NPU support with Wan2.2 MXFP4 and MindIE quantized attention paths, MOSS-TTS-Nano on Ascend A2, and multi-NPU MOSS-TTS Local deployment profiles. (#7210, #7561, #7192, #7052)
  • Enabled HunyuanImage-3.0 text-to-image and the LTX-2 family on Intel XPU, and made AR-diffusion KV preallocation device-agnostic. (#7674, #7961, #7777)
  • Expanded ROCm engine, executor, and entrypoint coverage, and enabled LTX DiffVAE kernel fusions on NVIDIA SM100/SM103. (#7397, #7398, #7350)

Breaking Changes

  • diffusion_batch_size has been removed. Use max_num_seqs instead. Unknown diffusion configuration fields now raise validation errors rather than being silently discarded. (#6484, #5172)
  • Duplex clients must follow the updated session contract. /v1/duplex is an alias of /v1/realtime?duplex=1; obsolete experimental client parameters and the native_duplex session parameter have been removed. Use auto_response instead of the removed full_duplex alias. (#7413, #7647, #8024)
  • Realtime audio and transcript events now use current, non-beta OpenAI event names. Clients consuming the previous names must update their handlers. (#7339, #7426)
  • First-party support for Dynin-Omni and dots.tts has been removed. (#7655)

Note

  • Qwen3-TTS's default Model Runner V2 remains experimental and CUDA-specific; non-CUDA platforms retain V1. Set model_runner: v1 to opt out. Its event-driven orchestration default is also model-specific and can be overridden with VLLM_OMNI_EVENT_DRIVEN_ORCH. (#7930, #7088)
  • Multi-process API serving is opt-in and currently limited to supported local EngineCore pipelines; diffusion stages and remote/headless stages are not supported in this mode. (#6923)
  • Native Mooncake KV transfer is opt-in. The existing transfer path remains available when kv_transfer_config is absent. (#7166)
  • Mage-VL uses its Transformers/reference implementation, while the initial JoyAI native speech profile executes stages sequentially. These integrations should not be interpreted as native vLLM execution for Mage-VL or overlapping streaming execution for JoyAI. (#6537, #5352)

What's Changed

New Contributors

Full Changelog: v0.28.0...v0.30.0

Check out latest releases or
releases around vllm-project/vllm-omni v0.30.0

Don't miss a new vllm-omni release

NewReleases is sending notifications on new releases.

Get notifications