sentence-transformers 6.0.0

v6.0.0 - MultiVectorEncoder for ColBERT & late interaction models, transformers v5, float32 scoring, faster training & encoding

on Python PyPI

This major release introduces Multi-Vector Embedding models, also known as late interaction or ColBERT-style models, as a fourth model type alongside SentenceTransformer, CrossEncoder, and SparseEncoder. Going forward, you'll be able to use Sentence Transformers for training, inferencing, and interpreting Multi-Vector Embedding models.

It also modernizes the dependency floors to transformers v5, fixes a class of silent scoring bugs caused by half precision, and speeds up both training and encoding.

Install this version with

# Training + Inference
pip install sentence-transformers[train]==6.0.0

# Inference only, use one of:
pip install sentence-transformers==6.0.0
pip install sentence-transformers[onnx-gpu]==6.0.0
pip install sentence-transformers[onnx]==6.0.0
pip install sentence-transformers[openvino]==6.0.0

# Multimodal dependencies (optional):
pip install sentence-transformers[image]==6.0.0
pip install sentence-transformers[audio]==6.0.0
pip install sentence-transformers[video]==6.0.0

# Or combine as needed:
pip install sentence-transformers[train,onnx,image]==6.0.0

MultiVectorEncoder: ColBERT-style late interaction models (#3794)

Sentence Transformers v6.0 introduces MultiVectorEncoder, for ColBERT-style late interaction retrieval. Where a regular embedding model compresses a whole text into one vector, a multi-vector model keeps one vector per token and scores query against document with the MaxSim operator. That preserves token-level matching information that a single vector has to average away, which usually means stronger retrieval at the cost of a bigger index. It is also the state of the art for visual document retrieval, where a text query is matched against page images directly, with no OCR step in between.

Any PyLate checkpoint and any Stanford-NLP ColBERT checkpoint loads straight into it, and colpali-engine models for visual document retrieval work too, through the same familiar API you already use for dense, sparse, and reranker models.

from sentence_transformers import MultiVectorEncoder

# Download from the 🤗 Hub
model = MultiVectorEncoder("lightonai/LateOn")

query_embeddings = model.encode_query(["Which planet is known as the Red Planet?"])
document_embeddings = model.encode_document([
    "Venus is often called Earth's twin because of its similar size and proximity.",
    "Mars, known for its reddish appearance, is often referred to as the Red Planet.",
    "Jupiter, the largest planet in our solar system, has a prominent red spot.",
    "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.",
])

print(query_embeddings[0].shape)
# (12, 128)

scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[10.7942, 11.1104, 10.9743, 11.0811]])

Mars wins, as it should, though notice how close the four scores are. That is normal for MaxSim: the scores often look similar, but the ranking is still exact. The blogpost explores this in more detail.

Note what you get back: a list of 2D tensors on the model device, one per input, each of shape (num_tokens, embedding_dim). Unlike dense embeddings, you cannot stack these into one rectangular tensor, because every input has its own token count. Pass convert_to_numpy=True for a list of numpy arrays instead, which is what you want once a corpus outgrows device memory.

Multi-vector models are also asymmetric: queries and documents go through different prefixes, different length caps, and different scoring masks. Unlike many dense models, where the two are interchangeable, encode_query and encode_document are required to get correct embeddings.

The MaxSim operator

Scoring uses MaxSim: for each query token, take its highest similarity against any document token, then sum those maxima across the query.

$$\text{MaxSim}(Q, D) = \sum_{Q_i \in Q} \max_{D_j \in D} Q_i \cdot D_j$$

You can read the operator as a soft alignment: every query token points at the one document token that best explains it, and the score is how well the document explains the query overall. The alignment does not have to be lexical, since the token embeddings are contextualized. But when an exact match does matter to you (a product code, a surname, a function name), MaxSim has a token sitting right there to match it, where a single-vector model had to fold it into an average.

Because MaxSim sums over query tokens, its magnitude scales with the query token count, so scores are not comparable across models with different query recipes. If you want scores on a bounded scale, use similarity_fn_name="meanmaxsim", which divides by the query token count and gives you an average cosine similarity in [-1, 1].

Scoring builds a 4-dimensional intermediate of every query token against every document token, which is the largest tensor in the operation. Every scoring function takes a chunk_elements budget that bounds it, defaulting to 100 million elements (roughly 400 MB in float32), so lower it if you run out of memory. Scores and gradients are bit-identical whatever you set it to. maxsim and maxsim_pairwise also take a device, which scores one chunk at a time on that device and moves each result straight back, letting you score a corpus larger than your VRAM on the GPU. Both are reachable through similarity, which forwards any extra keyword arguments to the scoring function:

scores = model.similarity(query_embeddings, document_embeddings, chunk_elements=1_000_000, device="cuda")

When training, pass the budget to the loss instead, with similarity_fct=partial(colbert_scores, chunk_elements=1_000_000). It chunks the document axis, so it composes with the loss-level score_mini_batch_size, which chunks the query axis.

Are they any good?

lightonai/LateOn and lightonai/DenseOn were trained by LightOn on the same data with the same ModernBERT backbone and the same 149M parameters, differing only in whether they keep one vector per token or pool down to one per document. Running both over all 13 NanoBEIR datasets isolates what that choice buys:

Late interaction wins on 9 of the 13 datasets and on the mean, by roughly one NDCG point. The four it loses (ArguAna, FiQA2018, SCIDOCS, and SciFact) are the shape of the tradeoff you should expect: a real gain in retrieval quality at the same model size, paid for in index footprint, rather than a universal win on every dataset. The same pair scores 57.22 against 56.20 on the full 15-dataset BEIR, a comparable gap, so the margin is not an artifact of the small benchmark.

That footprint is the real cost. One vector per token instead of one vector per document is a lot more vectors, only partly offset by the smaller dimension. Encoding 4,874 Natural Questions passages with lightonai/LateOn produced 608,414 token vectors, an average of 124.8 per passage:

That is about 42x the storage of the MiniLM index. Token Pooling cuts the vector count before any of that, real late interaction indexes compress heavily (the same vectors take 88 MB as a fast-plaid PLAID index), and using a multi-vector model as a reranker over a dense first stage avoids building an index at all.

Every checkpoint format loads

Multi-vector checkpoints have been published in several formats over the years. MultiVectorEncoder reads all of them, so loading looks the same whatever the model started life as:

from sentence_transformers import MultiVectorEncoder

# Native Sentence Transformers checkpoints. PyLate builds on the same schema,
# so any PyLate checkpoint loads identically
model = MultiVectorEncoder("lightonai/LateOn")
model = MultiVectorEncoder("mixedbread-ai/mxbai-edge-colbert-v0-17m")
model = MultiVectorEncoder("LiquidAI/LFM2-ColBERT-350M")

# Any Stanford-NLP ColBERT checkpoint, detected via the `HF_ColBERT` architecture
# marker. The inline projection weight and the recipe come from `artifact.metadata`
model = MultiVectorEncoder("colbert-ir/colbertv2.0")
model = MultiVectorEncoder("answerdotai/answerai-colbert-small-v1")

# transformers-native *ForRetrieval ports (ColPali, ColQwen2, ...)
model = MultiVectorEncoder("vidore/colqwen2-v1.0-hf")

# A bare transformer: a fresh random projection is appended, so training is required
model = MultiVectorEncoder("answerdotai/ModernBERT-base")

The recipe knobs that differ per checkpoint (marker prefixes for queries and documents, length caps, whether queries are padded out with [MASK] tokens, and which tokens are skipped when scoring documents) all live in the module configs, so print(model) shows you exactly what you loaded:

model = MultiVectorEncoder("colbert-ir/colbertv2.0")
print(model)
"""
MultiVectorEncoder(
  (0): Transformer({..., 'document_length': 180,
                    'query_expansion': {'strategy': 'fixed', 'attend': False, 'token': None, 'length': 32}})
  (1): Dense({'in_features': 768, 'out_features': 128, 'bias': False, ...})
  (2): MultiVectorMask({'skiplist_words': ['!', '"', '#', ...], 'skiplist_tasks': ['document'], ...})
  (3): Normalize({...})
)
"""

Following the design principle of the rest of the library, this behavior lives in swappable modules rather than in the model class: a Transformer producing contextualized token embeddings, a token-level Dense projecting each of them down, a MultiVectorMask deciding which tokens count during scoring, and a token-level Normalize.

Supported models

These are the checkpoints we test against directly, ranked by retrieval quality. The sentence-transformers tag on the Hub is the list that stays current, and for text retrieval in particular, any PyLate or Stanford-NLP ColBERT checkpoint loads whether or not it carries the tag yet. Where a revision is listed, pass it until the pull request on that repository is merged.

Text retrieval (29 models). NanoBEIR is the mean NDCG@10 over the 13 NanoBEIR datasets, a fast proxy for English text retrieval quality. A - means the model was not evaluated on it, which is the case for the non-English models.

Visual document retrieval (22 models). These embed page images as documents and text as queries. NanoViDoRe is the equivalent proxy over the ViDoRe benchmark subsamples.

Note that NanoBEIR and NanoViDoRe are small benchmarks, so their scores are not a substitute for evaluating on your own data, which is always the right way to pick a model.

Visual, audio, and video document retrieval

Late interaction is the state of the art for visual document retrieval: matching a text query against page images, with charts, tables, and layout intact, and no OCR step. This is what the ColPali family of models does, and those checkpoints run through the same API. Image documents are passed as URLs, local paths, or PIL images:

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("vidore/colqwen2.5-v0.2")

queries = [
    "What is the variable represented on the y-axis of the graph?",
    "Total outlay is maximum in which year?",
]
images = [
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc1.jpg",
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc2.jpg",
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc3.jpg",
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc4.jpg",
]

# A page yields far more vectors than a query: one per image patch
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(images)
print(query_embeddings[0].shape, document_embeddings[0].shape)
# (25, 128) (755, 128)

scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[13.8672, 12.3115, 12.1670, 11.0293],
#         [ 7.2012, 14.7207,  6.9414,  6.9746]])

The code is unchanged from the text case. Underneath, the processor handles the visual prompt and the image patches, and MaxSim scores query text tokens against document image patches. Page images are not the only non-text modality either: text, images, audio, and video are all accepted, and a checkpoint supports whichever of those its processor does, which model.modalities reports.

Because MaxSim is a sum of per-query-token maxima, a ranking decomposes exactly: every point of a document's score belongs to one query token and one document token. The new sentence_transformers.multi_vector_encoder.interpretability module overlays that decomposition onto the page as the standard ColPali heatmap, either aggregated over the query or one map per query token.

Token pooling

If the index footprint worries you, the most effective knob is to store fewer token vectors. HierarchicalTokenPooling implements the token pooling technique from Clavié, Chaffin, and Adams: it clusters each document's token vectors with Ward linkage on cosine similarity and replaces each cluster with its mean, keeping roughly 1 / pool_factor of the tokens.

from sentence_transformers import MultiVectorEncoder
from sentence_transformers.multi_vector_encoder.modules import HierarchicalTokenPooling

model = MultiVectorEncoder("lightonai/LateOn")
pooling = HierarchicalTokenPooling(pool_factor=2)

# 1. Per encode call
document_embeddings = model.encode_document(documents, token_pooling=pooling)

# 2. Standalone, on embeddings you already have saved
pooled = pooling.pool(document_embeddings)

# 3. Baked into the model, so every consumer of the checkpoint gets pooled documents
model.append(HierarchicalTokenPooling(pool_factor=2))
model.save_pretrained("my-pooled-colbert")

By default, pooling applies to documents only, since queries are short and are the side you cannot afford to distort. On the Natural Questions corpus above, the reduction tracks pool_factor closely:

The original experiments measured the retrieval cost of this on BEIR and found very little of it: 100.6% of the unpooled performance on average at pool_factor=2, and 99.0% at pool_factor=3. How much it costs on your data is corpus-specific, so measure it with an evaluator before you settle on a factor.

Update Stats

Introducing MultiVectorEncoder has been one of the largest updates to Sentence Transformers, introducing all of the following:

• Code:
  • New Trainer, Training Arguments, Data Collator, Model Card generation + template
  • 4 new modules: MultiVectorMask, BaseTokenPooling, HierarchicalTokenPooling, and LambdaTokenPooling
  • 4 new losses, including cached and knowledge distillation variants
  • 5 new evaluators
  • 4 new similarity functions (maxsim, maxsim_pairwise, mean_maxsim, mean_maxsim_pairwise) plus 6 named ColBERT scorers and 5 XTR scoring entry points
  • 1 new interpretability module for MaxSim heatmaps and similarity maps
  • ONNX and OpenVINO backend support, alongside half precision, Flash Attention, and torch.compile
• Tests:
  • 332 tests for MultiVectorEncoder loading, inference, training, etc., including parity checks against reference scores for 27 pretrained checkpoints
• Docs:
  • All new Training Overview, Loss Overview, and API Reference docs
  • 5 new usage documentation pages
  • 5 new training example documentation pages backed by 14 runnable example scripts
  • A pretrained models page covering 29 text retrieval and 22 visual document retrieval checkpoints
  • Migrating from PyLate and from colpali-engine guides

Resources

• Blogpost:
  • Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers
• Documentation:
  • Multi-Vector Encoder > Usage
  • Multi-Vector Encoder > Pretrained Models
  • Multi-Vector Encoder > Creating Custom Models
  • Multi-Vector Encoder > Speeding up Inference
  • Multi-Vector Encoder > Training Overview
  • Multi-Vector Encoder > API Reference
• Models:
  • Multi-vector models on the Hub

🚨 transformers v5, torch 2.2, and new dependency floors (#3794)

Sentence Transformers v6.0 requires transformers v5. The v4.x compatibility branches have been removed, which is what allows the new modality handling, chat template support, and unpadding paths to be relied upon rather than feature-detected. The floors that moved:

requires-python is unchanged at >=3.10. Note that multi-GPU training with streaming (IterableDataset) datasets needs accelerate>=1.13.0 in practice.

🚨 Higher-precision scoring (#3892, #3893, #3924, #3926)

Half precision ties too many scores together to rank with. Three separate places where that mattered are now computed in float32.

Reranker scores are the big one. CrossEncoder.predict (and rank) now upcast the logits to float32 before applying the activation function. A sigmoid in bfloat16 saturates and collapses the top candidates onto a handful of tied values, which randomizes their order. Measured on cross-encoder/ettin-reranker-32m-v1 in bfloat16 over three NanoBEIR datasets with 100 candidates per query:

NanoMSMARCO NDCG@10 alone goes from 0.0965 to 0.7093. If you run a half precision reranker with the default sigmoid activation, its ranking was essentially randomized before this release. Models using activation_fn=nn.Identity() (raw logits) were unaffected, as bf16 logits keep enough relative spacing.

Similarity scores from model.similarity / similarity_pairwise and the cos_sim family are now computed in float32 for float16 and bfloat16 embeddings. With 10,000 realistic cosine scores (mean 0.7, standard deviation 0.05), float32 keeps 9,983 distinct values where float16 keeps 593 and bfloat16 keeps just 93. bfloat16 can represent only 129 distinct values in the whole of [0.5, 1.0).

MaxSim sums over query tokens, reaching magnitudes where the bfloat16 grid is 0.125 wide, so maxsim and maxsim_pairwise accumulate the per-token maxima in float32 and always return float32 scores. The 4-dimensional scoring intermediate stays in the input dtype, so this does not change peak memory.

Note that encode() output dtypes are unchanged. Only the scoring step is upcast. For CrossEncoder.predict, the returned dtype changes only with convert_to_tensor=True or convert_to_numpy=False, as the default numpy output was already float32.

Separately, the multi-vector bf16 benchmarks were re-measured under this float32 accumulation (#3924). Most of the previously reported bf16 quality drop came from the scoring accumulation rather than from the embeddings: plain bf16 now sits at 99.0% of fp32 retrieval quality (was 95.0%), and bf16 with FlashAttention-2 is indistinguishable from fp32 at 99.96% (was 97.9%).

🚨 Other breaking changes (#3794, #3927, #3935)

• similarity and similarity_pairwise are methods, not properties. Calls like model.similarity(embeddings1, embeddings2) work unchanged, but assigning a custom function to model.similarity is no longer supported: it now silently shadows the method where it previously raised an AttributeError. Set model.similarity_fn_name = "dot" instead, which updates both. Note also that model.similarity.__name__ is now "similarity" rather than the resolved function name, which affected loss get_config_dict() output and generated model cards. The new sentence_transformers.util.similarity_fct_name() resolves it properly and the losses use it.
• A bare list of chat message dictionaries is now one conversation. model.encode([{"role": "user", ...}, {"role": "assistant", ...}]) produces one embedding, where v5.x read it as a batch of two inputs. Wrap each conversation in its own list to encode a batch: model.encode([[msg1], [msg2]]). This applies to SentenceTransformer, SparseEncoder, and MultiVectorEncoder. CrossEncoder is unaffected.
• Custom module classes require trust_remote_code=True (#3935). Loading a model whose modules.json references a class outside sentence_transformers executes third-party code, and a local directory no longer implies trust. This closes the bypass reported in #3801 and completes the deprecation cycle announced in v5.6 and v5.7. Unmet, it raises a ValueError naming the class and pointing at the repository or local path to inspect. Trainer checkpoint reloading (load_best_model_at_end, resume_from_checkpoint) keeps working for programmatically built models without the flag.
• quantize_embeddings returns a list of per-input matrices when given a list of 2D arrays, where it previously stacked them into one 3D array. Update callers that indexed the stacked array. An empty list now returns [] instead of raising, and a (0, dim) matrix returns a correctly shaped empty result.
• Multi-process encode(pool=..., precision="int8") now quantizes once after merging the worker results, so the calibration ranges match single-process encoding. Quantized indexes built with v5.x multi-process encoding are not bit-compatible and should be regenerated. Peak memory is higher, because the full float32 matrix is materialized before quantization.
• CrossEncoder.rank returns Python floats (#3927) as its "score" values, where it previously returned numpy.float32 scalars or 0-dimensional tensors. The results are directly JSON serializable, matching semantic_search. convert_to_numpy and convert_to_tensor on rank are now deprecated no-ops: call predict directly if you want an array or a tensor. Beyond the cleaner output, this avoids a device synchronization per comparison when sorting, which took 212ms for 1000 CUDA scalars against 0.089ms for Python floats.
• Normalize moved to sentence_transformers.base.modules. Existing models load fine and silently, but a model saved by v6.0 with a Normalize module cannot be loaded by Sentence Transformers older than v6.0.
• Cross-family conversion no longer inherits inference settings. Loading a SentenceTransformer checkpoint as a CrossEncoder (or any other such conversion) no longer picks up the source's prompts, default_prompt_name, similarity_fn_name, truncate_dim, or activation_fn, as those describe a model you are not loading. A reranker's default prompt being prepended to every encode call was the motivating case. Explicit keyword arguments still win. These conversions are now also logged at warning level, so they are visible at default verbosity.
• SimilarityFunction.possible_values() now includes "maxsim" and "meanmaxsim". Setting an unsupported similarity_fn_name on SentenceTransformer or SparseEncoder raises immediately rather than failing later, and a new SUPPORTED_SIMILARITY_FN_NAMES class attribute documents what each model type accepts.
• Automatically generated model cards now open with "It maps inputs to a N-dimensional dense vector space" rather than "It maps sentences & paragraphs to ...", since the model could be handling other modalities.

Faster training and encoding (#3938, #3794)

Multi-column losses now run one forward pass over merged columns (#3938). A training batch arrives as one feature dict per column (anchor, positive, negative_1, and so on), and the classic pattern runs the model once per column. The SentenceTransformer and SparseEncoder losses now pad and concatenate the like-width candidate columns into a single batch, keeping the anchor on its own forward pass since a 12-token query padded into 256-token documents costs more than it saves:

Loss trajectories match, up to dropout sampling. Losses fall back to per-column forward passes whenever the columns cannot be merged safely, for example with differing feature keys, disagreeing prompts or router tasks, or flattened Flash Attention inputs. The cached losses keep using GradCache, and AdaptiveLayerLoss opts out.

Backend benchmarks were re-measured for all four model types, with new Flash Attention columns and rewritten recommendations. For SentenceTransformer, float16 with Flash Attention and unpadding is now the fastest GPU configuration at 3.87x over float32, and ONNX on GPU is no longer recommended for short texts as float16 now beats it. For CrossEncoder, Flash Attention is explicitly not recommended, as unpadding does not apply to classification heads. For SparseEncoder, plain float16 remains the recommendation even though FA2 unpadding is now supported. See Speeding up Inference for the flowcharts.

Models can declare their dependency versions (#3934)

Model authors can now record which package versions their checkpoint needs, and loading verifies them up front instead of failing in a confusing way later. Add a requirements mapping to config_sentence_transformers.json, using PEP 440 specifiers:

{
    "model_type": "SentenceTransformer",
    "requirements": {
        "transformers": ">=5.15",
        "peft": {
            "specifier": ">=0.18,<0.20",
            "reason": "Older versions ignore the key_mapping, which silently randomizes the adapter weights."
        }
    }
}

Loading that model in an environment that does not satisfy it raises an ImportError listing every unmet requirement at once, with the optional reason included and a ready-to-run install command:

The model 'tomaarsen/my-model' requires:
- transformers>=5.15, but transformers==5.4.0 is installed.
- peft>=0.18,<0.20, but peft==0.17.0 is installed. Older versions ignore the key_mapping, which silently randomizes the adapter weights.
Install compatible versions with:
    pip install -U "transformers>=5.15" "peft>=0.18,<0.20"

"python" and "pytorch" are understood as special names, prereleases are accepted so nightlies and .dev0 builds do not trip the check, and anything unparsable warns and is skipped rather than blocking the load. It works for all four model types. See Declaring Version Requirements for details.

Evaluator and loss correctness (#3794, #3944, #3937)

• Pooling(include_prompt=False) no longer corrupts repeated forward passes (#3944). The pooling module used to write its prompt-excluded mask back into features["attention_mask"], but that key is what the encoder attends over on the next forward pass, and it is where the prompt boundary is read from. Any loss that embeds the same feature dicts twice therefore got a different answer each time. AdaptiveLayerLoss is the headline victim: on a model with a 3-token prompt, two consecutive calls with identical inputs returned 2.706679 and then 10.334954, where it is now stable at 1.777709. DenoisingAutoEncoderLoss was hit from another angle, handing its decoder an all-zero cross-attention mask. If you trained AdaptiveLayerLoss on an include_prompt=False model with prompts, your results will move. As a side effect, encode(output_value=None) now reports the full mask the encoder used, matching the input_ids and token_embeddings in the same dictionary. sentence_embedding and output_value="token_embeddings" are bit-identical.
• TripletEvaluator and SparseTripletEvaluator embed anchors with encode_query and positives and negatives with encode_document, instead of encode for all three. This is a no-op for models without query / document prompts, but asymmetric models will report different triplet accuracy than in v5.x, since their prompts, router routes, and per-task length caps are now applied. Both also now reject unknown similarity_fn_names and unknown margin keys at construction, where a typo previously degraded silently to a missing metric or a zero margin.
• InformationRetrievalEvaluator breaks score ties by corpus id, making its metrics independent of corpus_chunk_size. Previously torch.topk(..., sorted=False) plus a heap comparison made tie retention depend on chunk boundaries and favor larger corpus ids. Metrics change only where exact ties exist, such as duplicate documents or quantized embeddings. Inherited by the sparse and NanoBEIR variants.
• DistillKLDivLoss and SparseDistillKLDivLoss gained per-side temperatures (student_temperature, teacher_temperature) following the DenseOn and LateOn recipes, plus validation that catches previously silent misuse: non-positive or non-finite temperatures, fewer than three columns (a softmax over one candidate is constant, so the loss and its gradient are identically zero), and teacher score shapes that do not match the candidate columns. A new one-time warning reports how many teacher scores underflowed to exactly zero and recommends a teacher_temperature floor.
• XTR scoring validates its inputs (#3937): top_k must be positive, integer embeddings are upcast rather than producing an integer score grid, and a query padding mask is inferred from all-zero rows when none is given, matching the document side. XTR also now computes its Z normalizer as the paper's retrieval count rather than a positive-maxima proxy.
• Teacher labels are detached in the distillation losses across all model types, and MultipleNegativesRankingLoss rejects a NaN scale rather than accepting it.

Bug Fixes

• Fix NoDuplicatesBatchSampler silently no-opping on media datasets in #3794: PIL images and torchcodec decoders stringify to a fresh object address on every access, so every row looked unique and no duplicates were ever detected. Large numpy arrays had the opposite problem, as their truncated string representation made distinct arrays collide. Values are now keyed by content. Batches change for datasets with image, audio, video, or array columns. Plain text and numeric datasets are byte-identical.
• Fix xxhash 4.0 compatibility in #3928: xxh64_intdigest no longer accepts str, which crashed training with BatchSamplers.NO_DUPLICATES_HASHED (or precompute_hashes=True). Strings are now encoded before hashing. Digests are unchanged, so precomputed hashes stay valid.
• Fix pixel_values and other base-model arguments being silently dropped for PEFT models in #3794: PeftModel.forward hid the wrapped model's parameters, so the forward argument allowlist was built from the wrapper. It now unwraps first.
• Fix task and num_images_per_sample leaking into the transformers forward pass as unexpected keyword arguments in #3794, via an explicit denylist of Sentence Transformers internal feature keys that yields to a model actually declaring them.
• Fix trainer.evaluate() for VLM losses in #3794 by no longer gating media count tracking on self.training.
• Warn when spawned dataloader workers are not persistent in #3794: on Windows and macOS, every spawned worker imports Sentence Transformers before it can produce a batch, and without dataloader_persistent_workers=True that cost is paid again every epoch and every evaluation, commonly making training slower than dataloader_num_workers=0. The examples now use dataloader_num_workers=2 with persistent workers.
• Warn when a dataset column looks like an ID column in #3794, since it will be tokenized and trained on as text. The new sentence_transformers.util.resolve_ids resolves ID columns against lookup datasets, replacing PyLate's KDProcessing for knowledge distillation data.
• Support Flash Attention unpadding for SPLADE models in #3794, via a SpladePooling path that pools flattened sequences directly. Note that plain float16 remains the SparseEncoder recommendation, as unpadding reaches 2.3x to 2.4x against float16's 2.5x.
• Allow Dense to load configs containing unknown keys in #3794, dropping them with a warning instead of raising, so newer or foreign saves remain loadable.

Examples, Documentation, and Notebooks

• Align the MultiVectorEncoder documentation with the other three model types in #3933, adding the modality support sections, tabbed training overviews, and a pretrained models page rebuilt to 29 text and 22 visual document retrieval checkpoints, ranked by their NanoBEIR and NanoViDoRe scores in #3945.
• Add a v6 breaking changes section, a "Migrating from PyLate" guide, and a "Migrating from colpali-engine" guide to the Migration Guide in #3794 and #3926.
• Re-measure the inference benchmarks for all four model types in #3794, adding Flash Attention configurations and switching methodology to the median speedup at each backend's best batch size, then re-measure the multi-vector bf16 numbers under float32 MaxSim accumulation in #3924.
• Add a Module.on_model_ready hook in #3794 for modules that need model-dependent state after construction, documented alongside the other module extension points.
• Add query_length and document_length to Transformer, applying per-task tokenization caps in preprocess, and surface both in the generated model cards.

All Changes

• [chore] Increment dev version after release by @tomaarsen in #3914
• [tests] Update slow pretrained tests for transformers v5.6+ by @tomaarsen in #3911
• [v6] Add support for MultiVectorEncoder models by @tomaarsen in #3794
• Re-measure MVE bf16 benchmark quality under fp32 MaxSim accumulation by @tomaarsen in #3924
• [v6] fix: compute similarity scores in float32 to avoid low-precision ties (HPS) by @KisuYang in #3892
• [v6] fix: upcast cross-encoder logits to float32 before activation (HPS) by @KisuYang in #3893
• docs: Add v6 migration notes for the float32 scores, fix a semantic_search docstring by @tomaarsen in #3926
• [v6] Forward similarity kwargs to similarity functions, expose MaxSim device & chunking by @Samoed in #3905
• [fix] Encode strings before hashing for xxhash 4.0 compatibility by @tomaarsen in #3928
• [v6] refactor: return Python floats from CrossEncoder.rank by @tomaarsen in #3927
• [v6] unify document_chunk_elements / pair_chunk_elements into a single chunk_elements by @tomaarsen in #3931
• [v6] Expose chunk_elements on the ColBERT scorers by @tomaarsen in #3932
• docs: Align MultiVectorEncoder pages with the other archetypes by @tomaarsen in #3933
• [feat] Allow models to declare required dependency versions, verified on load by @tomaarsen in #3934
• [v6] Require trust_remote_code=True for custom module classes, dropping implicit local-directory trust by @tomaarsen in #3935
• [v6] Fix XTR input validation, padding masks, and integer embedding support by @eSVeeF in #3937
• [v6] Extend the merged column forward to the SentenceTransformer losses by @tomaarsen in #3938
• [v6] Keep the prompt-excluded mask out of the feature dicts by @tomaarsen in #3944
• [v6] Return tensors on the model device from MultiVectorEncoder encoding by @tomaarsen in #3943
• Speed up multi-vector padding and numpy to tensor conversion by @tomaarsen in #3942
• docs: Rank the pretrained MultiVectorEncoder models by NanoBEIR and NanoViDoRe by @tomaarsen in #3945
• docs: Point cross-references at their documented targets by @tomaarsen in #3947
• tests: Fix the pretrained MultiVectorEncoder tests for device tensors and Hub changes by @tomaarsen in #3949

New Contributors

• @KisuYang made their first contribution in #3892

Special Thanks

I especially want to thank the following teams and individuals for their contributions to this release, in no particular order:

• LightOn, and Antoine Chaffin (@NohTow), Raphaël Sourty (@raphaelsty), Paulo Moura, and Amélie Chatelain in particular, for building PyLate and fast-plaid, for being receptive to absorbing PyLate into Sentence Transformers, and for reviewing this work throughout. MultiVectorEncoder would not look like this without them.
• Omar Khattab and Matei Zaharia, for ColBERT, which everything here descends from.
• The ColPali team (Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo), for bringing late interaction to page images, and the ViDoRe team for helping get their configurations onto the Hub.
• Kisu Yang (@KisuYang), whose Reliable Evaluation Protocol for Low-Precision Retrieval (ACL 2026) is what surfaced the half precision scoring problems fixed in #3892 and #3893.
• Benjamin Clavié, Antoine Chaffin, and Griffin Adams, whose token pooling work HierarchicalTokenPooling implements.
• The core MTEB team, Kenneth Enevoldsen and Roman Solomatin (@Samoed, who also contributed #3905 to this release) among many others, for the kind of hidden work that keeps information retrieval research running.
• Answer.AI, mixedbread, LiquidAI, VAGO solutions, Perplexity, and everyone else publishing multi-vector checkpoints, for the models this release was tested against.

Apologies if I forgot anyone.

Full Changelog: v5.7.0...v6.0.0