vLLM Metal v0.28.0

v0.28.0 aligns the plugin release number with its vLLM v0.28.0 core dependency. The release contains 152 merged pull requests from 28 contributors since vLLM Metal v0.2.0.

Highlights

• No compiler required for release installs: the vllm-metal wheel contains its native extensions and precompiled Metal libraries, and the installer uses a prebuilt macOS arm64 vLLM wheel (#423, #508).
• Distributed serving: Metal gained vLLM Ray-executor support for its validated single-node configuration, pipeline parallelism, and dense data-parallel serving with one replica per Mac (#417, #427, #451).
• Speculative decoding: draft-model and n-gram proposers landed, Gemma 4 MTP gained multi-step drafting, and draft KV became scheduler-managed with bounded cold-ingest paths (#447, #456, #658, #630, #643).
• GGUF serving: MLX-native GGUF loading expanded from local Q8_0/Q4_0 models to Q4_1, Llama/Mistral mappings, and remote GGUF references (#453, #480, #498, #546, #635).
• M5 NAX prefill: an automatic paged-attention prefill kernel uses M5 NAX tensor units for supported MHA/GQA/MQA shapes. The reported serving workloads measured 21 to 41 percent lower mean TTFT and 27 to 33 percent higher total throughput where attention was a material part of the step (#623).
• Hybrid prefix caching: Qwen3.5-style SDPA + GDN models can restore recurrent-state checkpoints from scheduler block IDs when prefix caching is enabled (#584).
• Sampling and prefill projection: the deferred decode pipeline gained native temperature/top-k/top-p sampling for supported batches, while paged prefill projects vocabulary logits only for rows that will be sampled (#649, #590).

Installation

Requirements:

• Apple Silicon with macOS 15 or newer.
• Native arm64 Python 3.12.

Stable channel:

curl -fsSL https://raw.githubusercontent.com/vllm-project/vllm-metal/main/install.sh | bash -s -- --stable

The installer downloads prebuilt vllm-metal and vLLM wheels. It does not invoke a C++ or Metal compiler.

Release Artifacts

The release attaches a native Python 3.12 wheel for macOS 15 arm64:

vllm_metal-0.28.0-cp312-cp312-macosx_15_0_arm64.whl

The compatible upstream core is the prebuilt vLLM v0.28.0 macOS arm64 wheel.

Engine, Cache, and Performance

• The decode path gained one-step-ahead sampling, batched paged-prefill sampling, and batched structured-output mask application (#588, #582, #587).
• GDN prefill and decode reduced state handoff overhead through deferred scatter, removed per-layer materialization barriers, and in-place state-row updates (#620, #632, #634).
• Paged decode fused K/V cache writes and reused KV reads across speculative verification windows (#469, #534).
• Metal memory admission now honors --gpu-memory-utilization and rejects scheduler layouts larger than the allocated cache pool (#514, #527).
• Native non-greedy sampling remains opt-in and retains the torch path for seeded requests, penalties, logprobs, token constraints, and mixed batches (#649).

Models and Serving Features

• LoRA support expanded to QLoRA adapters on AWQ and MLX-quantized bases (#351, #470).
• PaddleOCR-VL gained multimodal and multi-image serving support (#422, #429).
• EXAONE 4.0 and the Laguna model family were added (#459, #562).
• Qwen3.8 hybrid models were documented and validated on Metal (#609).
• BGE-M3 gained dense embedding and sparse token-classification pooling through the encoder backend (#612).

Compatibility and Operational Changes

• vllm-metal and vLLM core are both versioned 0.28.0 for this release.
• MLX is pinned to exactly 0.32.0, matching the prebuilt native extension ABI.
• The prebuilt wheel deployment target is macOS 15.
• The optional Rust frontend was removed (#490).
• Source builds remain available through VLLM_METAL_BUILD_FROM_SOURCE; release installs use packaged native artifacts (#540).
• Development and stable releases now use separate channels. Stable releases are draft-first and tied to releases/vX.Y.Z branches (#652).

Known Boundaries

• The distributed modes above retain the topology and feature restrictions documented in their linked PRs; they do not imply unrestricted combinations of DP, PP, TP, LoRA, multimodal, and speculative decoding.
• Hunyuan Q/K normalization attribute names are recognized, but the linked change documents a remaining RoPE/norm ordering mismatch and is not a claim of complete Hunyuan support (#659).

Full changelog: v0.2.0...v0.28.0