deepset-ai/haystack v3.4.0-rc1

on GitHub

⭐️ Highlights

🐍 Python 3.15 Support, Python 3.10 Dropped

Haystack now officially supports Python 3.15. Python 3.10 reaches end of life in October 2026, so the minimum supported version is now 3.11. ⚠️

📦 ToolResultOffloadCompactor: Compact Agent Context Without Losing Tool Results

The experimental ToolResultOffloadCompactor is a new compaction strategy for CompactionHook. When compaction is triggered, it doesn't drop or summarize older tool results. It writes them to a ToolResultStore and leaves a reference in their place. The Agent can read the full output back on demand with a read tool scoped to the same store. Fresh results stay in context until compaction kicks in, so short runs pay nothing extra.

from typing import Annotated

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.hooks.compaction import CompactionHook, ToolResultOffloadCompactor
from haystack.hooks.tool_result_offloading import FileSystemToolResultStore
from haystack.tools import tool

store = FileSystemToolResultStore(root="tool_results")

@tool
def read_offloaded_result(path: Annotated[str, "Path of an offloaded tool result"]) -> str:
    """Read back the full content of an offloaded tool result."""
    content = store.read(path)
    if isinstance(content, bytes):
        return f"'{path}' holds {len(content)} bytes of binary content and cannot be read as text."
    return content

agent = Agent(
    chat_generator=OpenAIResponsesChatGenerator(model="gpt-5.4-nano"),
    tools=[web_search, read_offloaded_result],
    hooks={"before_llm": [CompactionHook(compactor=ToolResultOffloadCompactor(store=store), context_window=400_000)]},
)

⬆️ Upgrade Notes

• The page_number metadata field now means the same thing in every splitter: the page the chunk's text starts on. Two cases change:

  • RecursiveDocumentSplitter reported the page a chunk ended on. A chunk spanning a page break now reports the page it starts on, matching DocumentSplitter and MarkdownHeaderSplitter.
  • DocumentSplitter, MarkdownHeaderSplitter and EmbeddingBasedDocumentSplitter reported the previous page for a chunk that began with one or more form feeds (\f), even though all of the chunk's text was on the following page. Such chunks now report the page their text is on. A chunk made up entirely of page breaks is an empty page and keeps the page it starts on, so blank pages emitted by split_by="page" still get their own number.

  If you display page_number to users or store it alongside embeddings, re-index documents that contain form feeds to pick up the corrected values.

• RecursiveDocumentSplitter with split_unit="word" and split_overlap greater than 0 now returns chunks copied from the document as they are. Previously it rebuilt them from their words joined by single spaces, which added spaces the document does not have, collapsed line breaks and repeated spaces, and could drop form feeds (\f). The overlap also counts words the way split_length does, as text separated by spaces, so where words are separated only by line or page breaks, chunks can end at different words than before. Re-index documents split this way if you need their chunks to match the source text.

• RecursiveDocumentSplitter now keeps the whitespace between sentences when sentence_splitter_params sets keep_white_spaces to False. That is the sentence tokenizer's default, so this applies whenever sentence_splitter_params is passed without "keep_white_spaces": True. The tokenizer leaves that whitespace out, and the chunks used to glue their sentences together, as in "One two.Three four.": the glued words counted as one toward split_length, page breaks (\f) between sentences were lost, and split_idx_start and page_number fell behind the chunk's real start. Chunks are now copied from the document whatever keep_white_spaces is set to; it only changes the rules the tokenizer uses to decide where a sentence ends. Re-index documents split with this setting to pick up the corrected chunks.

• Dropped support for Python 3.10 since it reaches End of Life (EOL) in October 2026. The minimum supported Python version is now 3.11. If you are on Python 3.10, upgrade your Python version before upgrading Haystack.

• FilterPolicy.MERGE now intersects initialization and runtime logical filters when both use OR or NOT. Callers that relied on the previous broader result should construct a single explicit OR filter containing all alternatives and pass it with FilterPolicy.REPLACE instead.

• ChatMessage.from_openai_dict_format now raises a ValueError for content parts it can't convert: images referenced by a URL such as https://example.com/image.png (only images embedded as base64 data URLs, like data:image/png;base64,..., are supported), files referenced by file_id, and other part types such as input_audio. Previously, such messages were accepted because the whole list was stored as a single text, and ChatMessage.to_openai_dict_format sent that list back unchanged, so forwarding them to OpenAI worked by accident. You're affected if the OpenAI messages you convert contain any of these parts. To keep sending images by URL, download them and build the message yourself, for example ChatMessage.from_user(content_parts=["Describe this image", ImageContent.from_url(url)]). Send files inline as base64 file_data instead of referencing them by file_id. Extra keys on content parts, such as cache_control, are no longer passed through, so prompt caching through OpenRouter or LiteLLM stops for these messages.

• HTMLToDocument now defaults to encoding=None, delegating HTML encoding detection to Trafilatura. Set encoding="utf-8" to retain explicit UTF-8 decoding.

• Previously, HTMLToDocument used the encoding value in a source ByteStream's metadata to decode its content, overriding the converter's encoding setting. This metadata no longer affects decoding. If you relied on it, pass the encoding to the converter instead, for example HTMLToDocument(encoding="latin-1").

• The default model for OpenAITextEmbedder, OpenAIDocumentEmbedder, AzureOpenAITextEmbedder, and AzureOpenAIDocumentEmbedder has been changed from text-embedding-ada-002 to text-embedding-3-small.

  text-embedding-3-small is roughly 5x cheaper per token than the previous generation text-embedding-ada-002 and scores higher on the MTEB benchmark (see OpenAI's announcement).

  To preserve previous behavior, pass model="text-embedding-ada-002" explicitly (or, for the Azure variants, azure_deployment="text-embedding-ada-002").

  Note for Azure users: azure_deployment refers to the deployment name in your Azure OpenAI resource, not a model identifier. If you do not already have a text-embedding-3-small deployment in Azure, you will need to create one or pass the name of an existing deployment explicitly.

  Embeddings generated with the new default model are not compatible with embeddings produced by text-embedding-ada-002. If you have a document store populated with previously-generated embeddings, either keep using the old model explicitly or re-embed your corpus.

• The base Tool.warm_up() and Toolset.warm_up() methods were removed. OpenAI/Azure Chat Generators no longer warm up tools.

  If you use tools through an Agent, no changes are needed. For manual preparation, use warm_up_tools or warm_up_tools_async. Lazy toolsets such as MCPToolset must be warmed up before passing them to a Chat Generator.

🚀 New Features

• Added the experimental ToolResultOffloadCompactor for CompactionHook. Once compaction is triggered, it writes older tool results to a ToolResultStore and leaves a reference in their place, so the Agent can read the full output back with a read tool you scope to the same store. Fresh results stay in context until then.

  from typing import Annotated
  
  from haystack.components.agents import Agent
  from haystack.components.generators.chat import OpenAIResponsesChatGenerator
  from haystack.hooks.compaction import CompactionHook, ToolResultOffloadCompactor
  from haystack.hooks.tool_result_offloading import FileSystemToolResultStore
  from haystack.tools import tool
  
  store = FileSystemToolResultStore(root="tool_results")
  
  
  @tool
  def read_offloaded_result(path: Annotated[str, "Path of an offloaded tool result"]) -> str:
      """Read back the full content of an offloaded tool result."""
      content = store.read(path)
      if isinstance(content, bytes):
          return f"'{path}' holds {len(content)} bytes of binary content and cannot be read as text."
      return content
  
  
  # web_search stands in for your own tool that returns long results.
  hook = CompactionHook(compactor=ToolResultOffloadCompactor(store=store), context_window=400_000)
  agent = Agent(
      chat_generator=OpenAIResponsesChatGenerator(model="gpt-5.4-nano"),
      tools=[web_search, read_offloaded_result],
      hooks={"before_llm": [hook]},
  )

⚡️ Enhancement Notes

• When an Agent runs inside a Pipeline, the anonymous pipeline telemetry event now also reports the chat generator class and its tools: the number of tools, the class and name of each tool, the component wrapped by each ComponentTool, and the classes of any Toolset objects.

• Haystack now officially supports Python 3.15.

• Removed the Sockets.__getattribute__ override. It looked up a _sockets attribute that no instance has, so on every attribute access it raised and caught AttributeError and never returned a socket; socket access has always gone through the copies __init__ and __setitem__ put in __dict__. Without the override, accessing a socket attribute such as component.__haystack_input__.query is about 14x faster.

• HTMLToDocument now uses Trafilatura's encoding detection instead of assuming UTF-8, allowing non-UTF-8 sources to be converted without manually specifying their encoding. If conversion produces incorrect characters, install trafilatura[all] for additional encoding detection support, or set the converter's encoding explicitly.

• LLMEvaluator fallback warnings now name the component class, for example FaithfulnessEvaluator, and say that the result for a failed input is set to None. A response that isn't valid JSON or is missing the expected keys now gets its own warning with the response and the parsing error.

• LLMRanker fallback warnings for an empty query, a chat generation failure and a response parsing failure now consistently say the deduplicated input documents are returned unranked, so the fallback order isn't mistaken for a ranking.

• QueryExpander fallback warnings now say what is returned: only the original query, or no queries when include_original_query is False. QueryExpander now also warns when the ChatGenerator response isn't valid JSON, including the response and the parsing error, and when the response contains no valid queries.

• When no chat_generator is passed, LLMRanker and QueryExpander now use OpenAIResponsesChatGenerator instead of OpenAIChatGenerator, with the same model and temperature. The default generator sets "store": False so OpenAI does not keep the requests and responses.

• Agent now uses its close()/close_async() methods to release its configured tools and supports asynchronous tool preparation using warm_up_async(). Invoking a tool directly through tool.invoke() or tool.invoke_async() automatically calls its warm-up methods when available.

  This enables tools to manage synchronous and asynchronous resources separately. We'll update tools in Haystack Core Integrations to take advantage of this support.

• When a tool call made by Agent fails, its haystack.agent.step.tool tracing span now carries the tag haystack.tool.error set to True. Tracers can use it to mark the span as failed, so a failed call is no longer indistinguishable from a tool that returns an {"error": ...} dict. The tag is set even when content tracing is disabled.

🐛 Bug Fixes

• Fixed DOCXToDocument dropping the fragment of an external link when Word stores it separately in the w:anchor attribute instead of the hyperlink address. For example, a link to https://example.org/manual#refund-policy was converted to https://example.org/manual with both the markdown and the plain link_format option. The complete link target, including the fragment, is now preserved.
• Fixed RecursiveDocumentSplitter assigning page_number values that drifted upwards, and could exceed the number of pages in the document, when split_overlap was greater than 0. A page break repeated in an overlapping tail was counted once per chunk it appeared in. page_number is now derived from the chunk's offset in the source document instead of a running counter, so a repeated page break cannot be counted twice.
• Fixed RecursiveDocumentSplitter reporting split_idx_start values past the chunk's real start when split_unit="word" and split_overlap was greater than 0, which also pushed page_number onto pages the chunk had not reached yet. split_idx_start now points at the chunk's start in the source document, and the _split_overlap ranges cover exactly the text the two chunks share.
• Fixed RecursiveDocumentSplitter dropping the whitespace after the last sentence of each piece of text it split into sentences, such as the "\n\n" ending a paragraph longer than split_length. This lost text, and moved the split_idx_start of every later chunk before its real start, which could put a chunk on the page before the one it starts on. The whitespace is now kept at the end of the chunk it follows, so with no overlap the chunks add up to the document again. Such a chunk can exceed split_length by that trailing whitespace.
• bound_event_dict_values now also bounds log values nested inside list, tuple and dict fields. Exceptions in a container are rendered with str instead of repr, so a UnicodeDecodeError no longer writes the whole buffer it failed to decode into the log line, and containers are cut short in place with [container budget exhausted], [recursive container] or [maximum depth reached] markers when they reach the traversal limits.
• Converting PDF pages to images, as PDFToImageContent and DocumentToImageContent do, now closes the pypdfium2 document even when rendering a page raises an error. Before, the document was only closed after all pages rendered successfully, so a failing page left it open until garbage collection.
• DOCXToDocument now extracts text and tables inside text boxes. Previously this content was silently dropped.
• With Python 3.15 lazy imports, from haystack.tools import tool now returns the tool decorator. Before, lazy from haystack.tools import tool and imports under PYTHON_LAZY_IMPORTS=all returned the haystack.tools.tool module instead.
• AnswerExactMatchEvaluator, DocumentMAPEvaluator, DocumentMRREvaluator and DocumentRecallEvaluator now raise a descriptive ValueError when called with empty input lists, instead of failing with an opaque ZeroDivisionError. This matches the existing behavior of DocumentNDCGEvaluator.
• Fixed FilterPolicy.MERGE returning documents that the initialization filters exclude. When the initialization and runtime filters were both logical filters sharing an OR or NOT operator, their conditions were concatenated, which unions the two filters rather than applying both. They are now combined under an AND so each filter still restricts the result. Merging two AND filters is unchanged.
• SentenceSplitter now reads its curated abbreviation lists as UTF-8 regardless of the system locale. Previously, on Windows, the German list was decoded with the ANSI code page (for example cp1252), so abbreviations containing umlauts or ß, such as ggü. or außenpol., were garbled and did not prevent sentence splits when extend_abbreviations=True.
• Fixed a SchemaGenerationError when using @tool, create_tool_from_function, or ComponentTool with a parameter typed as Annotated[State, ...], including optional variants. These parameters are now excluded from the schema sent to the language model and receive the Agent's live State object when the tool runs, just like parameters typed as State.
• InMemoryDocumentStore no longer counts Documents without content, such as image Documents that only have an embedding, in its BM25 statistics. Previously they increased the document count and lowered the average document length, which changed the BM25 scores and ranking of text Documents even though BM25 retrieval never returns Documents without content.
• ChatMessage.from_openai_dict_format now converts messages whose content is a list of content parts. Previously, the whole list was stored in a single TextContent, so ChatMessage.text returned a list instead of a string. In user messages, text parts now become TextContent, image_url parts with base64 data URLs become ImageContent, and file parts with inline file_data become FileContent. In system, developer, and assistant messages, text parts are joined with a newline into a single text. An assistant's refusal, given as a refusal part or as the refusal field, is stored in ChatMessage.meta["refusal"] instead of the text, and ChatMessage.to_openai_dict_format sends it back. content that is neither a string nor a list now raises a ValueError instead of being stored as text.
• EvaluationRunResult.aggregated_report, detailed_report and comparative_detailed_report with output_format="csv" now write the CSV file as UTF-8. Previously the file used the locale encoding, so on Windows a report with text outside the ANSI code page (e.g. Japanese or Cyrillic questions) was left half-written and an error message was returned instead, and accented characters were written as cp1252.
• Fixed expand_page_range silently dropping pages when a reversed range (e.g. "7-5") appeared alongside valid entries. Mixed inputs like ["1-3", "7-5", "8"] previously returned [1, 2, 3, 8] with no warning, losing pages 5-7. A reversed range now raises ValueError with a descriptive message identifying the offending range.
• logger.exception() on loggers from haystack.logging.getLogger now logs the active traceback, as the standard library does. Previously it logged only the message, without the stack trace, in both JSON and console output. Passing exc_info explicitly still works.
• MetaFieldRanker now applies missing_meta when the meta values can't be sorted together, for example 1 and "x". In this case the Documents keep their original order, but Documents missing the meta field are dropped with "drop", placed first with "top", and placed last with "bottom". Previously, all Documents were returned in their original order and missing_meta was ignored.
• OpenAIChatGenerator and OpenAIResponsesChatGenerator no longer drop a tool call whose arguments are an empty string, and no longer raise TypeError when they are null, in non-streaming responses. OpenAI-compatible servers such as vLLM send these for a tool with no parameters. Both now become empty arguments, as they already did in streaming responses and in ChatMessage.from_openai_dict_format.
• The default prompt of QueryExpander now states that its examples show the output for 3 requested queries and that exactly n_expansions queries must be returned. The trailing JSON format example, which always showed three queries, is removed. Before, the examples showed three or four queries without saying so, which contradicted the instruction for any other n_expansions value.
• Fixed SentenceWindowRetriever returning context_windows with the chunks in the wrong order when the documents have no split_idx_start meta field, for example chunks produced by MarkdownHeaderSplitter. The chunks were concatenated in the order returned by the Document Store instead of by split_id, so the context passed to an LLM could be scrambled while context_documents was correctly sorted. The chunks are now sorted by split_id before being merged, in both run and run_async.
• JsonSchemaValidator.run() now raises a ValueError saying the message list is empty when it receives no messages, instead of an IndexError from reading the last message.
• ToolResultOffloadHook and ToolResultOffloadCompactor no longer stop the Agent run when a ToolResultStore write fails, for example when FileSystemToolResultStore rejects a key containing .. or the disk is full. The result stays in the conversation, a warning naming the tool and the store is logged, and the remaining results are still offloaded. If one part of a result with several content blocks fails to write, the whole result stays in the conversation.
• Fixed misleading LLMDocumentContentExtractor messages. The failure warning said "Skipping metadata extraction", and a document that couldn't be converted to an image got extraction_error set to "Document has no content". The docstring also named the wrong metadata field (content_extraction_error instead of extraction_error). Warnings now say the document is returned in failed_documents, and a response that isn't a JSON object is now logged too.
• Fixed the LLMEvaluator warning that summarizes failed inputs. It logged the raw template LLM evaluator failed for {errors} out of {len(list_of_input_names_to_values)} inputs. instead of the counts. This also affects FaithfulnessEvaluator and ContextRelevanceEvaluator.
• LostInTheMiddleRanker now validates all input documents before deduplication and top_k selection, raising ValueError if any document is non-textual. Previously, single-document selections and documents excluded by top_k could bypass the text-only validation.
• Fix Pipeline.connect() silently skipping a new socket-to-socket connection when other connections already link the same sender and receiver components. Repeating an existing connection remains a no-op, and invalid additional connections now raise the expected error.
• When a pipeline fails and its snapshot can't be written to disk, error.pipeline_snapshot_file_path is now None. Before, it held the path of the snapshot file that was never written.
• CSVToDocument in row mode now preserves values from a CSV column named row_number as csv_row_number in metadata (with a suffix on collision), instead of overwriting them with the generated row index.
• Redact large base64-encoded binary payloads (ImageContent and FileContent) nested inside ToolCallResult when tracing is enabled. Replaces raw base64 data with placeholder strings (such as "Base64 string (N characters)"), preventing multi-megabyte payloads from leaking into pipeline tracing spans.
• Raise FilterError instead of leaking a Python TypeError when an ordering metadata filter compares a datetime with a non-datetime value.
• OpenAIResponsesChatGenerator and AzureOpenAIResponsesChatGenerator now map the raw reasoning text that the Responses API returns as reasoning_text content to ReasoningContent.reasoning_text, both when streaming (response.reasoning_text.delta events) and when not streaming. Previously, this text was dropped while streaming and only kept in ReasoningContent.extra["content"] otherwise. OpenAI models only return reasoning summaries, which still take precedence, but other providers such as OpenRouter return the raw reasoning. When such a reply is sent back to the model in a later turn, the raw reasoning is no longer duplicated as a summary.

💙 Big thank you to everyone who contributed to this release!

@Aditya17-bot, @agnieszka-m, @amengnew, @anakin87, @ArzelaAscoIi, @Ayushdevo, @basilalshukaili, @Ben160804, @bhaskargurram-ai, @bilgeyucel, @bogdankostic, @camgrimsec, @chrikrah, @davidsbatista, @Evihut, @hardness1020, @Harsh23Kashyap, @julian-risch, @kacperlukawski, @L4XB, @LimbC-C, @LindseyZ1205, @Marcuswang0824, @Morningstar202604, @otiscuilei, @rautaditya2606, @rxchitrx, @ShousenZHANG, @sjrl, @Timotty, @TimurRakhmatullin86, @Vedant-Agarwal, @yuv009