⬆️ Upgrade Notes
DocumentNDCGEvaluatornow matches documents by theircontentfield by default instead of their auto-generatedid. Previously, ground truth and retrieved documents were matched only if they had identicalidvalues, which rarely happened in practice since IDs are generated independently for each Document instance. As a result, NDCG scores computed with this evaluator may change for existing pipelines. To keep the previousid-based matching behavior, passdocument_comparison_field="id"when constructing the evaluator.
🚀 New Features
- Added native asynchronous support (
run_async) toLLMEvaluator,FaithfulnessEvaluator, andContextRelevanceEvaluator. This allows concurrent evaluation loops inside async applications like FastMCP or FastAPI without blocking the main event loop, while automatically falling back to thread workers for synchronous chat generators. - Added optional YAML frontmatter extraction to
MarkdownToDocument. When initialized withextract_frontmatter=True, YAML frontmatter at the beginning of a Markdown file is removed from the converted content and added toDocument.meta.
⚡️ Enhancement Notes
-
Added
document_comparison_fieldparameter toDocumentNDCGEvaluator, consistent withDocumentMAPEvaluator,DocumentMRREvaluator, andDocumentRecallEvaluator. Users can now match documents by"content","id", or any"meta.<key>"field when calculating NDCG scores. -
Add
output_passthroughoption toConditionalRouter. Whenoutput_passthrough: Trueis set in a route, theoutputfield is treated as a plain variable name instead of a Jinja2 template, and the value is passed directly from the pipeline inputs to the route output. This allows routing of complex non-basic types such as dataclasses and Pydantic models without unwanted Jinja2 template processing.Without
output_passthrough, the router rendersoutputas a Jinja2 template, which converts the value to its string representation. Custom types cannot survive that round-trip:# Without output_passthrough — the object is silently converted to a string routes = [ { "condition": "{{True}}", "output": "{{query}}", "output_name": "out", "output_type": ParsedQuery, } ] router = ConditionalRouter(routes) result = router.run(query=ParsedQuery(text="hello", intent="search", entities=[])) # result["out"] == "ParsedQuery(text='hello', intent='search', entities=[])" # ^^^ str, not ParsedQuery — the object was destroyed
Set
output_passthrough: Trueto skip Jinja2 entirely and pass the value directly from kwargs:from haystack.components.routers import ConditionalRouter from dataclasses import dataclass, field @dataclass class ParsedQuery: text: str intent: str # "search" | "chat" entities: list[str] = field(default_factory=list) routes = [ { "condition": "{{query.intent == 'search'}}", "output": "query", # variable name, not a Jinja2 template "output_name": "search_query", "output_type": ParsedQuery, "output_passthrough": True, }, { "condition": "{{query.intent == 'chat'}}", "output": "query", "output_name": "chat_query", "output_type": ParsedQuery, "output_passthrough": True, }, ] router = ConditionalRouter(routes) query = ParsedQuery(text="What is Haystack?", intent="search", entities=["Haystack"]) result = router.run(query=query) assert isinstance(result["search_query"], ParsedQuery) # type preserved assert result["search_query"] is query # same object, no copying
-
Added an opt-in
expand_reference_rangesparameter toAnswerBuilder. When enabled, reference ranges like[6-10]and comma-separated ranges like[1-3,7-9]are expanded to the corresponding document indices in RAG answers. The feature is disabled by default to preserve existing parsing behavior.
⚠️ Deprecation Notes
-
AzureOCRDocumentConverteris deprecated and will be removed from Haystack in version 3.0. It is moving to theazure-form-recognizer-haystackpackage. To continue using it, install the package withpip install azure-form-recognizer-haystackand update your import as follows:from haystack_integrations.components.converters.azure_form_recognizer import AzureOCRDocumentConverter
-
DatadogTraceris deprecated and will be removed from Haystack in version 3.0. It is moving to thedatadog-haystackpackage. To continue using it, install the package withpip install datadog-haystackand add theDatadogConnectorcomponent to your pipeline to enable tracing, or update your import as follows:from haystack_integrations.tracing.datadog import DatadogTracer
Note that, starting with Haystack 3.0, Datadog tracing is no longer auto-enabled when
ddtraceis installed. Use theDatadogConnectorcomponent to enable it. -
OpenAIGenerator,AzureOpenAIGenerator,HuggingFaceAPIGenerator, andHuggingFaceLocalGeneratorhave been deprecated and will be removed in Haystack 3.0. Generators living in Haystack Core Integrations will also be removed in Haystack 3.0.Their chat counterparts (
OpenAIChatGenerator,AzureOpenAIChatGenerator,HuggingFaceAPIChatGenerator,HuggingFaceLocalChatGenerator) are the replacement. Starting from Haystack 2.30.0, all ChatGenerators also accept a plainstras input to make the transition easier.How to migrate:
Direct usage (running a generator from Python code)
Before:
from haystack.components.generators import OpenAIGenerator gen = OpenAIGenerator() result = gen.run("What is NLP?") text = result["replies"][0] # str meta = result["meta"][0] # dict with model metadata
After:
from haystack.components.generators.chat import OpenAIChatGenerator gen = OpenAIChatGenerator() result = gen.run("What is NLP?") # str input accepted directly reply = result["replies"][0] # ChatMessage text = reply.text # str meta = reply.meta # dict with model metadata (now on the message)
System prompt
Before:
from haystack.components.generators import OpenAIGenerator gen = OpenAIGenerator(system_prompt="You are concise.") result = gen.run("What is NLP?")
After:
from haystack.components.generators.chat import OpenAIChatGenerator from haystack.dataclasses import ChatMessage gen = OpenAIChatGenerator() result = gen.run([ ChatMessage.from_system("You are concise."), ChatMessage.from_user("What is NLP?"), ])
Pipeline usage
Pipelines that connected
PromptBuilder(output:str) to a legacy Generator continue to work unchanged when you swap in a ChatGenerator. The Haystack pipeline type system automatically convertsstrtolist[ChatMessage]at the connection edge.Before:
from haystack.components.generators import OpenAIGenerator from haystack.components.builders import PromptBuilder pipeline.add_component("prompt_builder", PromptBuilder(template=prompt_template)) pipeline.add_component("llm", OpenAIGenerator()) pipeline.connect("prompt_builder", "llm") # str -> str
After, minimal change (smart connection still works):
from haystack.components.generators.chat import OpenAIChatGenerator from haystack.components.builders import PromptBuilder pipeline.add_component("prompt_builder", PromptBuilder(template=prompt_template)) pipeline.add_component("llm", OpenAIChatGenerator()) pipeline.connect("prompt_builder", "llm") # str -> list[ChatMessage], auto-converted
Alternatively, use
ChatPromptBuilder:from haystack.components.generators.chat import OpenAIChatGenerator from haystack.components.builders import ChatPromptBuilder from haystack.dataclasses import ChatMessage template = [ChatMessage.from_user(prompt_template)] pipeline.add_component("prompt_builder", ChatPromptBuilder(template=template)) pipeline.add_component("llm", OpenAIChatGenerator()) pipeline.connect("prompt_builder.prompt", "llm.messages")
-
HuggingFaceAPIChatGenerator,HuggingFaceAPITextEmbedder,HuggingFaceAPIDocumentEmbedder, andHuggingFaceTEIRankerare deprecated and will be removed from Haystack in version 3.0. They are moving to thehuggingface-api-haystackpackage. To continue using them, install the package withpip install huggingface-api-haystackand update your imports as follows:from haystack_integrations.components.generators.huggingface_api import HuggingFaceAPIChatGenerator from haystack_integrations.components.embedders.huggingface_api import HuggingFaceAPIDocumentEmbedder from haystack_integrations.components.embedders.huggingface_api import HuggingFaceAPITextEmbedder from haystack_integrations.components.rankers.huggingface_api import HuggingFaceTEIRanker
-
DocumentLanguageClassifierandTextLanguageRouterare deprecated and will be removed from Haystack in version 3.0. They are moving to thelangdetect-haystackpackage. To continue using them, install the package withpip install langdetect-haystackand update your imports as follows:from haystack_integrations.components.classifiers.langdetect import DocumentLanguageClassifier from haystack_integrations.components.routers.langdetect import TextLanguageRouter
-
OpenAPIConnector,OpenAPIServiceConnectorandOpenAPIServiceToFunctionsare deprecated and will be removed from Haystack in version 3.0. They are moving to theopenapi-haystackpackage. To continue using them, install the package withpip install openapi-haystackand update your imports as follows:from haystack_integrations.components.connectors.openapi import OpenAPIConnector from haystack_integrations.components.connectors.openapi import OpenAPIServiceConnector from haystack_integrations.components.converters.openapi import OpenAPIServiceToFunctions
These components are a legacy way to connect Haystack to external APIs. For most use cases, consider using the
MCPToolinstead, which is the modern, standardized way to give pipelines and agents access to external tools and services. -
OpenTelemetryTraceris deprecated and will be removed from Haystack in version 3.0. It is moving to theopentelemetry-haystackpackage. To continue using it, install the package withpip install opentelemetry-haystackand add theOpenTelemetryConnectorcomponent to your pipeline to enable tracing, or update your import as follows:from haystack_integrations.tracing.opentelemetry import OpenTelemetryTracer
Note that, starting with Haystack 3.0, OpenTelemetry tracing is no longer auto-enabled when
opentelemetry-sdkis installed. Use theOpenTelemetryConnectorcomponent to enable it. -
SearchApiWebSearchis deprecated and will be removed from Haystack in version 3.0. It is moving to thesearchapi-haystackpackage. To continue using it, install the package withpip install searchapi-haystackand update your import as follows:from haystack_integrations.components.websearch.searchapi import SearchApiWebSearch
-
SentenceTransformersTextEmbedder,SentenceTransformersDocumentEmbedder,SentenceTransformersDocumentImageEmbedder,SentenceTransformersSparseTextEmbedder,SentenceTransformersSparseDocumentEmbedder,SentenceTransformersDiversityRanker, andSentenceTransformersSimilarityRankerare deprecated and will be removed from Haystack in version 3.0. They are moving to thesentence-transformers-haystackpackage. To continue using them, install the package withpip install sentence-transformers-haystackand update your imports as follows:from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersDocumentEmbedder from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersDocumentImageEmbedder from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersSparseTextEmbedder from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersSparseDocumentEmbedder from haystack_integrations.components.rankers.sentence_transformers import SentenceTransformersDiversityRanker from haystack_integrations.components.rankers.sentence_transformers import SentenceTransformersSimilarityRanker
-
SerperDevWebSearchis deprecated and will be removed from Haystack in version 3.0. It is moving to theserperdev-haystackpackage. To continue using it, install the package withpip install serperdev-haystackand update your import as follows:from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
-
The spacy backend of
NamedEntityExtractoris deprecated and will be removed from Haystack in version 3.0. It is moving to thespacy-haystackpackage and being renamed toSpacyNamedEntityExtractor. To continue using it, install the package withpip install spacy-haystackand update your import as follows:from haystack_integrations.components.extractors.spacy import SpacyNamedEntityExtractor
-
TikaDocumentConverteris deprecated and will be removed from Haystack in version 3.0. It is moving to thetika-haystackpackage. To continue using it, install the package withpip install tika-haystackand update your import as follows:from haystack_integrations.components.converters.tika import TikaDocumentConverter
-
TransformersZeroShotDocumentClassifier,TransformersTextRouter,TransformersZeroShotTextRouter,HuggingFaceLocalChatGenerator,NamedEntityExtractor, andExtractiveReaderare deprecated and will be removed from Haystack in version 3.0. They are moving to thetransformers-haystackpackage. Some of them are also being renamed. To continue using them, install the package withpip install transformers-haystackand update your imports as follows:from haystack_integrations.components.classifiers.transformers import TransformersZeroShotDocumentClassifier from haystack_integrations.components.routers.transformers import TransformersTextRouter from haystack_integrations.components.routers.transformers import TransformersZeroShotTextRouter # HuggingFaceLocalChatGenerator was renamed to TransformersChatGenerator from haystack_integrations.components.generators.transformers import TransformersChatGenerator # NamedEntityExtractor (Hugging Face backend) was renamed to TransformersNamedEntityExtractor from haystack_integrations.components.extractors.transformers import TransformersNamedEntityExtractor # ExtractiveReader was renamed to TransformersExtractiveReader from haystack_integrations.components.readers.transformers import TransformersExtractiveReader
-
LocalWhisperTranscriberandRemoteWhisperTranscriberare deprecated and will be removed from Haystack in version 3.0. They are moving to thewhisper-haystackpackage. To continue using them, install the package withpip install whisper-haystackand update your imports as follows:from haystack_integrations.components.audio.whisper import LocalWhisperTranscriber from haystack_integrations.components.audio.whisper import RemoteWhisperTranscriber
🔒 Security Notes
- Strengthened the template-variable sanitization in
ChatPromptBuilderto cover cases where specially crafted template variables could still be interpreted as structured content (e.g., images, tool calls) instead of plain text. Protection now relies on two independent mechanisms. - Fixed a path-traversal issue in
DocumentToImageContentandLLMDocumentContentExtractor. Whenroot_pathis set, afile_pathin document metadata that escapes the root (via../or an absolute path) now raises aValueErrorbefore any file is read, preventing attacker-controlled metadata from reading arbitrary files on the host. - The
repr()of aSecretbuilt viaSecret.from_token(...)no longer includes the raw token. Previously the dataclass-generated repr printed the token verbatim, so any path that formats a component's locals — exception tracebacks,print(), Jupyter echo, log aggregators — could leak the API key. The token is now rendered as<redacted>.
🐛 Bug Fixes
- Six retriever components (CogneeRetriever, FAISSEmbeddingRetriever, FalkorDBCypherRetriever, FalkorDBEmbeddingRetriever, MongoDBAtlasFullTextRetriever, ValkeyEmbeddingRetriever) were missing from the retriever index table in both current and versioned docs.
DocumentJoinernow raises a clearValueErrorwhen the providedweightssum to zero, instead of failing with an opaqueZeroDivisionErrorwhile normalizing them.- Fixed
document_matches_filterraising aFilterErrorwhen comparing a string date inDocument.metaagainst adatetimeobject using ordering operators (>,>=,<,<=). - Metadata filters that use an unsupported operator now raise a clear
FilterErrorthat lists the valid operators, instead of a crypticKeyError. For example, a typo such as{"field": "meta.year", "operator": "gt", "value": 2000}(instead of">") previously surfaced asKeyError: 'gt'. This applies to both comparison operators and logical operators (AND,OR,NOT). - Fixed a
TypeErrorinAnswerJoinerwhen running withsort_by_score=Trueand at least one answer had ascoreofNone. As documented, an answer without a score is now treated as if its score were negative infinity and sorted last, instead of raisingTypeError: '<' not supported between instances of 'float' and 'NoneType'. - Fixed a
ZeroDivisionErrorinInMemoryDocumentStore.bm25_retrievalthat occurred at query time when every stored document had empty (but notNone) content. Such a tokenless corpus has no vocabulary and an average document length of zero, which made all three BM25 algorithms divide by zero. Retrieval now scores every candidate as0.0in this case: unscaledBM25Okapireturns the documents with score0.0, whileBM25LandBM25Plusreturn an empty list (non-positive scores are filtered). Corpora with at least one non-empty document are unaffected. GeneratedAnswerandExtractedAnswernow serialize to the same flat format as all other Haystack dataclasses (Document,ChatMessage, etc.), removing an inconsistency that caused redundant type metadata to be embedded in pipeline snapshots andStateobjects. Deserialization is backward compatible:from_dict()still accepts the old{"type": "...", "init_parameters": {...}}format.- Fixed the
>=and<metadata filter operators returning wrong results for datetime strings that represent the same moment in time but are formatted differently (for example"2025-01-01"vs"2025-01-01T00:00:00", or the same instant expressed with different timezone offsets). Equality was previously checked by raw string comparison instead of comparing the parsed datetimes, so>=evaluated toFalseand<evaluated toTruefor equal datetimes. This caused boundary documents to be silently dropped from date range queries inInMemoryDocumentStoreand any other component relying ondocument_matches_filter. - Fixed a crash in
DocumentJoinerwhen usingjoin_mode="distribution_based_rank_fusion"with documents that havescore=None. The score normalization now treats a missing score as0(consistent with how the mean/standard-deviation are computed), instead of raisingTypeError: unsupported operand type(s) for -: 'NoneType' and 'float'. DocumentSplitternow raises a clearValueErrorat initialization whensplit_overlapis greater than or equal tosplit_length. Previously this configuration was accepted and only failed later at run time with an opaque error from the underlying windowing helper (the window step became zero or negative).- Fixed
FaithfulnessEvaluatorandContextRelevanceEvaluatorto exclude failed queries (scorenan) from the aggregate mean score instead of silently returningnan. A warning is now logged when one or more queries are skipped. - Fixed
FileTypeRoutersilently routing MIME types containing+(such asimage/svg+xml,application/ld+json, and everyapplication/*+xml/application/*+json) into theunclassifiedbucket. Entries inmime_typeswere being compiled as regex, so the+was interpreted as a quantifier and the type never matched itself. Each source's MIME type is now checked againstmime_typesby exact equality first, falling back to the existing regex match — so literal MIMEs route correctly and patterns likeaudio/.*keep working unchanged. - Fixed
GeneratedAnswer.from_dictraising anIndexErrorwhen the serializedmetacontained an emptyall_messageslist. Deserialization now guards against the empty list (consistent withto_dict) and round-trips correctly. - Fixes
HuggingFaceLocalGeneratorso that when multiplestop_wordsare configured the generator returns exactly N replies instead of N×M replies. Previously the list comprehension created a cross-product, duplicating replies and leaving some stop words unstripped. - Fixed
human_in_the_loopconfirmation strategies (_run_confirmation_strategiesand_run_confirmation_strategies_async) raising an unhandledKeyErrorwhen a tool call references a tool name that isn't in thetoolslist (e.g. the chat model hallucinated it). The call now passes through unmodified toToolInvoker, which already raises a cleanToolNotFoundException(respectingraise_on_failure) for this case. HTMLToDocumentnow skipsByteStreamobjects with empty content instead of passing them to trafilatura. This prevents noisy lxml parse error logs (such as "Document is empty") when a fetcher emits an empty stream.- Fixed a silent data-loss bug in
Pipeline.connect()where anylist[T] -> Tconnection was accepted without warning (e.g.list[Document] -> Document) and only the first list element was forwarded at runtime, dropping the rest. The conversion is now restricted to inner typesstrandChatMessage, matching the original intent; any otherlist[T] -> Twiring now raisesPipelineConnectErroratconnect()time. As a defense-in-depth measure, the runtimeUNWRAPconversion also raisesValueError(wrapped inPipelineRuntimeError) when given a multi-element list, so any future regression of the connect-time gate cannot silently drop items. This also affects e.g. a generator usingn>1whoserepliesare wired into a single-stror single-ChatMessageconsumer — that pipeline now raises instead of silently usingreplies[0]. - Fixed
LLMEvaluator.to_dictnot serializing theraise_on_failureparameter, causing it to silently reset toTrueafter a pipeline round-trip. - Fixed
InMemoryDocumentStore.load_from_diskreconstructing documents with the plainDocumentconstructor instead ofDocument.from_dict. Documents containing ablob(ByteStream) or asparse_embedding(SparseEmbedding) were loaded with those fields as raw dictionaries, which later crashedrepr(),to_dict(), equality comparison,save_to_diskof the reloaded store, and any component accessingdocument.blob.data. - Fixed
MetaFieldGroupingRankerraising aTypeErrorwhensort_docs_bypoints to a non-numeric metadata field (for example ISO date strings) and some documents are missing that field. Documents that miss the field are now placed at the end of their group, regardless of the field's type. - Add
EmbeddingBasedDocumentSplitterto the PreProcessors index page, and addMarkdownHeaderSplitterandSpacyNamedEntityExtractorto the sidebar navigation. These components had documentation pages but were missing from navigation, making them undiscoverable by users browsing the docs. - Update parameters in MultiRetriever to allow for more flexible retrieval of documents. Changes include:
top_k_per_retriever: Allows specifying the maximum number of documents to return per retrievertop_k: Allows specifying the maximum number of documents to be retrieved from combined results of all retrievers
- Fixed a
TypeErrorwhen filtering documents on a nested metadata field (e.g.meta.user.age) where an intermediate value is not a dictionary (for exampleNoneor a scalar). Such intermediate values are now treated as a missing field (None) instead of raising, which previously crashedfilter_documentsfor stores likeInMemoryDocumentStore. - Fixed
OpenAIResponsesChatGeneratordroppingencrypted_contentfrom reasoning items in streaming mode. Two issues were addressed:_convert_streaming_chunks_to_chat_messagewas discarding all reasoning extras exceptidandtype; and_convert_response_chunk_to_streaming_chunkhad no handler for theresponse.output_item.doneevent for reasoning items, which is the only event that carriesencrypted_content. Both are now fixed, soencrypted_contentis preserved end-to-end and multi-turn conversations withstore=Falseandinclude=["reasoning.encrypted_content"]work correctly. - Fixed
RecursiveDocumentSplittersilently ignoringsplit_overlapwhen none of the configuredseparatorsmatch the input text. In that code path the component fell back to fixed-size chunking but returned the result directly without calling_apply_overlap, so every chunk after the first was missing the expected overlap prefix. The fix applies_apply_overlapto the fallback chunks before returning them, making behaviour consistent with the separator-matched code path. All three split units ("char","word", and"token") are affected and are now covered by regression tests. - Fixed
RecursiveDocumentSplitterproducing incorrectsplit_idx_startmetadata values whensplit_unit="word"(or"token") is used together withsplit_overlap > 0. Previously,self.split_overlap(a word/token count) was subtracted directly from a character-offset counter, causing every chunk after the first to report a wrong start position in the original document. The fix uses the actual character length of the overlap suffix, obtained from_get_overlap(), so the offset is always correct regardless ofsplit_unit. - Fixed
RecursiveDocumentSplitternot populating the_split_overlapmetadata whensplit_unitis"word"or"token"andsplit_overlapis greater than 0. The overlap length was computed with a word/token count while the offsets it is compared against are character positions, so it became negative and the overlap metadata was silently dropped. The overlap is now measured in characters, matching the ranges it records. - Fixed
serialize_typedroppingNoneTypewhen it appears as a regular argument of atypinggeneric. Types such astyping.Dict[str, None],typing.Tuple[int, None],typing.List[None]and unions with more than two members liketyping.Union[str, int, None]were serialized incorrectly (for example totyping.Dict[str]), which produced malformed type strings that could not be deserialized again.typing.Optional[X]is still serialized astyping.Optional[X]without a redundant trailingNone. - Fixed
SuperComponent.runandSuperComponent.run_asyncraisingValueError: The truth value of ... is ambiguouswhenever a non-scalar input (numpy arrays, pandas DataFrames/Series, torch tensors, or any object whose__ne__returns a non-scalar) was passed. The internal sentinel filter is now an identity check (is not _delegate_default) instead of value equality (!= _delegate_default), so it no longer invokes__ne__on user-provided values. - Fixed two bugs in
TopPSamplerthat silently disabled filtering:- Documents with integer scores were treated as if they had no score at all, so the sampler logged a warning and returned all documents unfiltered. Integer scores are now handled like float scores (booleans are still rejected).
- Passing
top_p=0.0torun()was silently replaced by the value set in the constructor because of a falsy-value check. The override is now honored, returning the single highest-scoring document, consistent withTopPSampler(top_p=0.0).
- Fix
InputSocket.__post_init__crashing withIndexErrorwhen a Variadic input uses a bareIterablewithout a type argument (e.g.Annotated[Iterable, HAYSTACK_VARIADIC_ANNOTATION]). Now raisesComponentErrorwith a clear message instead of an unhelpfulIndexError. - Fixed serialization of
FallbackChatGeneratorto use the standardcomponent_to_dictserialization path. This ensures custom chat generator components without an explicitto_dictmethod are no longer silently omitted from the serializedchat_generatorslist. HierarchicalDocumentSplitternow validates itsblock_sizesandsplit_overlapparameters at construction time. Previously, an emptyblock_sizesset silently produced no splits, and asplit_overlaptoo large relative to the smallest block size raised a confusing error referencingsplit_length, a parameter that doesn't exist on this component.HierarchicalDocumentSplitternow raises a clearValueErrorin both cases, using its own parameter names and identifying the offending value.HTMLToDocumentandMarkdownToDocumentnow accept anencodingparameter (default"utf-8") and honourByteStream.meta["encoding"]at run time, mirroring the behaviour ofTextFileToDocument. Previously both converters hardcodeddecode("utf-8"), causingUnicodeDecodeErrorfor non-UTF-8 sources whose encoding was supplied viaByteStreammetadata.- Fixed
InMemoryDocumentStore.embedding_retrievalproducingNaNcosine similarity scores when a document (or the query) had a zero-norm embedding. Zero-norm vectors are now handled without dividing by zero, so such documents receive a score of0.0instead ofNaN(which silently corrupted ranking). - Preserve falsy metadata values such as
0,False, and empty strings whenmeta_fields_to_embedis used by sentence-transformer embedders and rankers. QueryExpandernow ignores malformed responses wherequeriesis not a list, so a string response is no longer split into single-character queries.
💙 Big thank you to everyone who contributed to this release!
@Aarkin7, @Amnah199, @anakin87, @ATOM00blue, @axelray-dev, @Ayushhgit, @bilgeyucel, @bogdankostic, @camgrimsec, @charliesheh, @davidsbatista, @ErenAta16, @Goodnight77, @GovindhKishore, @greymoth-jp, @gyx09212214-prog, @hoangsonww, @HotThoughts, @i-anubhav-anand, @immuhammadfurqan, @jdoughty04, @jialiuyang, @julian-risch, @Kunal-Somani, @milljer, @mustafakhokhar, @NIK-TIGER-BILL, @Osamaali313, @oyinkanchekwas, @rautaditya2606, @ritikraj2425, @RitwijParmar, @santino18727-debug, @shr8769, @sjrl, @spideyashith, @Srivatsa03, @SyedShahmeerAli12, @vedjaw, @vidigoat