github huggingface/transformers v5.21
Release: v5.21.0
on GitHub

3 hours ago

Release v5.21.0

New Model additions

MiniCPM-OCR

MiniCPM-OCR is a document-understanding model from OpenBMB's MiniCPM-V family. It keeps the MiniCPM-V 4.6 architecture, which pairs a SigLIP vision encoder and a window-attention merger with a Qwen3.5 language model. It is specialized for reading text-dense images such as scanned pages, screenshots, receipts and tables.

Links: Documentation | Paper

Flint

flint_benchmarks_nvfp4

Flint is Recursive's mixture-of-experts language model. Compared to a standard SwiGLU MoE transformer such as Qwen3-MoE, it uses hybrid attention (sliding-window attention with rotary embeddings over the current token and the previous 512 tokens in most layers, and full attention without positional encoding in the remaining layers). It also has learned attention sinks, as in GPT-OSS, with per-head RMS-normalized queries and keys. Each sparse layer routes a token to 6 of 256 experts by sigmoid score and adds an ungated shared expert, and the first two layers are dense.

Links: Documentation

UltraVBert

UltraVBert extends ModernVBert by pairing the UltraBERT text encoder with a SigLIP vision encoder for visual document understanding and retrieval. Its image processor splits each page into 512-pixel tiles plus a global view. The connector merges each 4 x 4 block of vision patches into one token, so a tile becomes 64 image tokens, and RMS-normalizes them to the scale of UltraBERT's token embeddings. The max_image_tiles setting (27 by default) caps the number of tiles per page.

Links: Documentation

Gradium Phonon TTS

Gradium Phonon TTS is Gradium's family of on-device text-to-speech models, trained with the Pocket TTS recipe. A causal transformer reads a voice prompt and the text, then emits one continuous 32-dimensional audio latent per step at 12.5 Hz through a one-step flow head, and an end-of-speech head decides when the utterance ends. The latents are decoded to 24 kHz audio by a Pocket TTS audio tokenizer. Unlike Pocket TTS, voice conditioning uses 512-dimensional latents from a separate, unreleased speaker codec, so voices come as precomputed latents and cannot be cloned from a recording.

Links: Documentation

Pocket TTS

Pocket TTS is Kyutai's 100M-parameter text-to-speech model, built as a Continuous Audio Language Model that runs faster than real time on CPU. A causal transformer emits one continuous audio latent per frame (12.5 Hz) through a one-step flow head, and an end-of-speech head decides when to stop. A Mimi-based audio tokenizer encodes voice prompts and decodes the generated latents to 24 kHz audio. Two flow heads are released, Lagrangian self-distillation (default) and drifting, and voices can be cloned from a recording, passed as precomputed latents, or loaded from cached KV states.

Links: Documentation | [Paper](https://huggingface.co/papers/2509.06926 (the Continuous Audio Language Models paper, arXiv 2509.06926). It is the only paper the documentation links, and it describes the architecture family Pocket TTS belongs to. The docs do not cite a separate Pocket TTS technical report.) | [Blog Post](NONE. The documentation links no official blog post. It points only to the original code repository at https://github.com/kyutai-labs/pocket-tts.)

Pocket TTS Audio Tokenizer

The Pocket TTS audio tokenizer is the codec used by Pocket TTS. It has Mimi's architecture (SEANet convolutions plus a small causal transformer on each side), but its residual vector quantizer is replaced by a continuous 32-dimensional bottleneck, the VAE-GAN from Continuous Audio Language Models. It converts 24 kHz audio into latents at 12.5 Hz and decodes them back to audio. It serves as the audio_tokenizer of PocketTTSProcessor, encoding voice prompts and decoding generated latents, and its encoder and decoder can also be loaded on their own.

Links: Documentation | Paper

Breaking changes

Fixed Gemma4 processor imports (the Gemma4, Gemma4 unified and EmbeddingGemma2 processors) so from transformers import * no longer fails in environments that have PIL but not torch; no user action is needed.

  • 🚨 Fix gemma4 cross imports resulting in import failing without torch but with PIL (#49748) by @Sai-Suraj-27

Added an "fp8" backend to QuantizedCache, alongside quanto and hqq, that stores KV states in torch.float8_e4m3fn with a single scale per tensor; enable it with cache_implementation="quantized" and cache_config={"backend": "fp8"}.

Tensor parallelism now has a dedicated embedding style, shards every model's input embeddings by default, and supports widths that aren't divisible by the TP size, so ColwiseParallel and RowwiseParallel now apply only to linear layers; custom TP plans that applied them to embeddings must switch to the new embedding style.

  • 🚨 TP: give embeddings their own style, shard input embeddings by default, and shard widths that don't divide by the TP size (#49570) by @IlyasMoutawwakil

Every MoE TP and EP plan now shards its experts, so models no longer keep all experts on every rank, and a new test_moe_parallel_plans_shard_experts test enforces this; custom or downstream MoE parallel plans must add expert sharding to pass it and to avoid wasted memory.

Cache

This release improves cache handling across the library. Flash Attention now works with static caches, BLOOM no longer crashes with static cache or compiled generate, chunked prefill supports cache continuation, and the Qwen4-Exp indexer's position history stays aligned with its cached keys. Linear-attention caches also gain new getters (is_recording_past, get_recurrent_state, conv-state access) that stop exposing cache internals and prepare for paged-c

Vision

Vision image processors now composite PNG transparency onto white instead of exposing black pixels, and background_color is preserved correctly across the DeepSeek-VL and Janus processors. Model fixes cover SmolVLM's inputs_merger when image tokens aren't a multiple of the patch size, HunYuan-VL image assignment during batched beam search, and Ernie4.5-VL-MoE expert sharding under TP/EP. The DETR

Generation

This release fixes a range of generation bugs: BLOOM crashes with static cache or compiled generate, generate dropping the type of dict-like inputs such as ModelOutput, and incorrect image assignment or placeholder handling in batched or beam-search generation for HunYuan-VL and Idefics2/3. It also fixes Qwen3-Omni audio generation mode handling, SAM mask-generation multi-zoom crops, empty frame s

Parallelization

This release extends parallelism support and fixes several related bugs. It adds a tensor-parallel plan for Nemotron and makes Ernie4.5-VL-MoE shard its text and vision experts under TP and EP. It also restores expert-parallel token-dispatch gradients on torch 2.7–2.10, fixes a Molmo2 model-parallel device mismatch, and corrects base_model_prefix in several model families (PP-OCR detectors

Quantization

Quantization now handles DeepSeek-V4.1's FP4 experts correctly: FineGrainedFP8Config gains an expert_dtype option ("fp8" by default, or "fp4") that the FP8 experts read instead of allocating them as FP8. TimesFM 2.5 no longer casts float inputs to an integer weight dtype when loaded with quantization such as bitsandbytes 4/8-bit. A broader cleanup

Bugfixes and improvements

Significant community contributions

The following contributors have made significant changes to the library over the last release:

  • @ydshieh
    • [tests_fetcher] fix missing .as_posix() in preprocessor_patterns path keys (#49994)
    • fix(ci): skip bot suggestion if a maintainer already posted run-slow: (#49992)
    • [Fetcher] Fix missing test deps for dynamically-resolved image processors (#49826)
    • Fix NameError in CI comment workflows: missing section marker variables (#49801)
    • [CI] Improve run-slow PR comments and remove CI recap (#49782)
    • Fix tests_fetcher backbone step (never worked since introduction) (#49742)
    • Fix SSH runner: add /opt/venv/bin to PATH for Python 3.11 images (#49759)
  • @remi-or
    • [Fix] Fix NUMA and cpuset detection in CB (#49980)
    • [Refactor] Make CB utils depend on no CB import (#49968)
    • [Fix] Remove duplicate assignment in DeepseekV3.2 config (#49896)
    • [CB] Reduce warmup time by not computing attention over full cache (#49853)
    • [CB] Reduce the max_memory_percent in case of a CPU model (#49844)
    • [modular] Extend the "del self.xxx" pattern to match typed assignments (#49735)
    • [Cache] Linear attn cache: use new getters (#49375)
    • [Cache] Get conv states (#49309)
    • [Cache] Add get_recurrent_state (#49306)
    • [Cache] Add is_recording_past (#49305)
    • [Kernels] Add MPS to supported device types (#49655)
    • Skip Muse TP (#49717)
    • [CB] Various small fixes for MPS (#49654)
  • @Nikhi00718
    • Composite transparency in shared RGB conversion and image processors (#49079)
  • @eustlb
    • [pocket_tts, gradium_phonon_tts] use sub_configs_defaults (#49868)
    • [gradium_phonon_tts] point to the released Hub repos (#49867)
    • [EmbeddingGemma2] update integration tests (#49576)
    • Add GradiumPhononTTS (#49791)
    • add PocketTTS (#49589)
    • add PocketTTSAudioTokenizer (#49588)
    • 🟡 [mimi] separate encoder and decoder (#49577)
    • mimi moshi output capturing (#49670)
    • fix moshi failing tests (#49750)
  • @jp1924
    • logits_to_keep bool mask for all ForXXX classes (#48825)
  • @molbap
    • Fix several object detection models/loader fix for module tying (#49827)
    • [CI] fix perceriver old regression (#49792)
    • [CI] no split modules for SAM (#49766)
    • Enable tests forgotten in idefics3 (#49758)
    • fix idefics3 generation issues (#49738)
    • fix tests for generation (#49740)
    • multi device fix for molmo2 (#49736)
  • @athrael-soju
  • @harshaljanjani
    • fix(vision): Route the DETR conversion scripts through from_pretrained (#49830)
    • fix(audio): Route the ALM conversion scripts through from_pretrained (#48525)
    • [vLLM] Add Qwen-Omni Thinker base models and Gemma 3n per-layer inputs (#49437)
  • @lovishmadaan
    • Add Flint model support. (#49824)
  • @tarekziade
    • Remove the resolved prototype Whisper MIC expected failure (#49785)
    • Update MIC fleet references and documentation (#49768)
    • Cleaned up tokenizer tests (#49669)
    • Fix test selection for virtual module imports (#49690)
    • Fix flaky Evolla tests producing NaN outputs (#49660)
  • @yonigozlan
    • Fix small bug in auto mappings (#49664)
  • @ArthurZucker
    • Quantization cleanpup (#49548)
Check out latest releases or
releases around huggingface/transformers v5.21

Don't miss a new transformers release

NewReleases is sending notifications on new releases.

Get notifications