Release Notes

v2.21.0-rc0

New Features

• Added the AzureOpenAIResponsesChatGenerator, a new component that integrates Azure OpenAI's Responses API into Haystack. This unlocks several advanced capabilities from the Responses API:

  • Allowing retrieval of concise summaries of the model's reasoning process.
  • Allowing the use of native OpenAI or MCP tool formats, along with Haystack Tool objects and Toolset instances.

  Example with reasoning and web search tool:

  from haystack.components.generators.chat import AzureOpenAIResponsesChatGenerator
  from haystack.dataclasses import ChatMessage  
  
  chat_generator = AzureOpenAIResponsesChatGenerator(azure_endpoint="https://example-resource.azure.openai.com/", azure_deployment="gpt-5-mini", generation_kwargs={"reasoning": {"effort": "low", "summary": "auto"}}, )  
  response = chat_generator.run(messages=[ChatMessage.from_user("What's Natural Language Processing?")] )
  
  print(response["replies"][0].text) 

• Add an extra field to ToolCall and ToolCallDelta to store provider-specific information.

• If logprobs are enabled in the generation kwargs, return logprobs in ChatMessage.meta for OpenAIChatGenerator and OpenAIResponsesChatGenerator.

• Added the OpenAIResponsesChatGenerator, a new component that integrates OpenAI's Responses API into Haystack. This unlocks several advanced capabilities from the Responses API:

  • Allowing retrieval of concise summaries of the model's reasoning process.
  • Allowing the use of native OpenAI or MCP tool formats, along with Haystack Tool objects and Toolset instances.

  Example with reasoning and web search tool:

  from haystack.components.generators.chat import OpenAIResponsesChatGenerator 
  from haystack.dataclasses import ChatMessage  
  chat_generator = OpenAIResponsesChatGenerator( model="o3-mini", generation_kwargs={ {"summary": "auto", "effort": "low"}     },tools=[{"type": "web_search"}] ) 
  response = chat_generator.run( messages=[ ChatMessage.from_user("What's a positive news story from today?")] )
  print(response["replies"][0].text) 

  Example with structured output:

from pydantic import BaseModel 
from haystack.components.generators.chat import OpenAIResponsesChatGenerator 
from haystack.dataclasses import ChatMessage  

class WeatherInfo(BaseModel):     
     location: str 
     temperature: float 
     conditions: str  


chat_generator = OpenAIResponsesChatGenerator(model="gpt-5-mini", generation_kwargs={"text_format": WeatherInfo} ) 

response = chat_generator.run(messages=[ChatMessage.from_user("What's the weather in Paris?")] ) 

• Updated our serialization and deserialization of PipelineSnapshots to work with pydantic BaseModels

• Added async support to SentenceWindowRetriever with a new run_async() method, allowing the retriever to be used in async pipelines and workflows.

• Added warm_up() method to all ChatGenerator components (OpenAIChatGenerator, AzureOpenAIChatGenerator, HuggingFaceAPIChatGenerator, HuggingFaceLocalChatGenerator, and FallbackChatGenerator) to properly initialize tools that require warm-up before pipeline execution. The warm_up() method is idempotent and follows the same pattern used in Agent and ToolInvoker components. This enables proper tool initialization in pipelines that use ChatGenerators with tools but without an Agent component.

• The AnswerBuilder component now exposes a new parameter return_only_referenced_documents (default: True) that controls if only documents referenced in the replies are returned. Returned documents include two new fields in the meta dictionary:

  • `source_index`: the 1-based index of the document in the input list

  - `referenced`: a boolean value indicating if the document was referenced in the replies (only present if the reference_pattern parameter is provided). These additions make it easier to display references and other sources within a RAG pipeline.

Enhancement Notes

• Adds generation_kwargs to the Agent component, allowing for more fine-grained control at run-time over the chat generation.
• Added a revision parameter to all Sentence Transformers embedder components (SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder, SentenceTransformersSparseDocumentEmbedder, and SentenceTransformersSparseTextEmbedder) to allow users to specify a specific model revision/version from the Hugging Face Hub. This enables pinning to a particular model version for reproducibility and stability.
• Updated the components Agent, LLMMetadataExtractor, LLMMessagesRouter, and LLMDocumentContentExtractor to automatically call self.warm_up() at runtime if they have not been warmed up yet. This ensures that the components are ready for use without requiring an explicit warm-up call. This differs from previous behavior where the warm-up had to be manually invoked before using these components otherwise they would raise a RuntimeError.
• Improve log-trace correlation for DatadogTracer by using the official ddtrace.tracer.get_log_correlation_context() method.
• Improved Toolset warm-up architecture for better encapsulation. The base Toolset.warm_up() method now warms up all tools by default, while subclasses can override it to customize initialization (e.g., setting up shared resources instead of warming individual tools). The warm_up_tools() utility function has been simplified to delegate to Toolset.warm_up().

Bug Fixes

• Fix deserialization of state schema when it is None in Agent.from_dict.

• Fixed type compatibility issue where passing list[Tool] to components with a tools parameter (such as ToolInvoker) caused static type checker errors. In version 2.19, the ToolsType was changed to Union[list[Union[Tool, Toolset]], Toolset] to support mixing Tools and Toolsets. However, due to Python's list invariance, list[Tool] was no longer considered compatible with list[Union[Tool, Toolset]], breaking type checking for the common pattern of passing a list of Tool objects.

  The fix explicitly lists all valid type combinations in `ToolsType`: Union[list[Tool], list[Toolset], list[Union[Tool, Toolset]], Toolset]. This preserves backward compatibility for existing code while still supporting the new functionality of mixing Tools and Toolsets.

  Users who encountered type errors like "Argument of type 'list[Tool]' cannot be assigned to parameter 'tools'" should no longer see these errors after upgrading. No code changes are required on the user side.

• When creating a pipeline snapshot we make sure to use _deepcopy_with_exceptions when copying component inputs to avoid deep copies of items like components and tools since they often contain attributes that are not deep-copyable. For example, the LinkContentFetcher has httpx.Client as an attribute which throws an error if we try to deep copy it.