Release Notes

v2.1.0-rc1

Highlights

Add the "page_number" field to the metadata of all output documents.

⬆️ Upgrade Notes

• The HuggingFaceTGIGenerator and HuggingFaceTGIChatGenerator components have been modified to be compatible with huggingface_hub>=0.22.0.

  If you use these components, you may need to upgrade the huggingface_hub library. To do this, run the following command in your environment: `bash pip install "huggingface_hub>=0.22.0"`

🚀 New Features

• Add SentenceTransformersDiversityRanker. The Diversity Ranker orders documents in such a way as to maximize the overall diversity of the given documents. The ranker leverages sentence-transformer models to calculate semantic embeddings for each document and the query.

• Adds truncate and normalize parameters to HuggingFaceTEITextEmbedder and HuggingFaceTEITextEmbedder for allowing truncation and normalization of embeddings.

• Add trust_remote_code parameter to SentenceTransformersDocumentEmbedder and SentenceTransformersTextEmbedder for allowing custom models and scripts.

• Add a new ContextRelevanceEvaluator component that can be used to evaluate whether retrieved documents are relevant to answer a question with a RAG pipeline. Given a question and a list of retrieved document contents (contexts), an LLM is used to score to what extent the provided context is relevant. The score ranges from 0 to 1.

• Add DocumentMAPEvaluator, it can be used to calculate mean average precision of retrieved documents.

• Add DocumentMRREvaluator, it can be used to calculate mean reciprocal rank of retrieved documents.

• Add a new FaithfulnessEvaluator component that can be used to evaluate faithfulness / groundedness / hallucinations of LLMs in a RAG pipeline. Given a question, a list of retrieved document contents (contexts), and a predicted answer, FaithfulnessEvaluator returns a score ranging from 0 (poor faithfulness) to 1 (perfect faithfulness). The score is the proportion of statements in the predicted answer that could by inferred from the documents.

• Introduce HuggingFaceAPIChatGenerator. This text-generation component uses the ChatMessage format and supports different Hugging Face APIs: - free Serverless Inference API - paid Inference Endpoints - self-hosted Text Generation Inference.

  This generator will replace the HuggingFaceTGIChatGenerator in the future.

• Introduce HuggingFaceAPIDocumentEmbedder. This component can be used to compute Document embeddings using different Hugging Face APIs: - free Serverless Inference API - paid Inference Endpoints - self-hosted Text Embeddings Inference. This embedder will replace the HuggingFaceTEIDocumentEmbedder in the future.

• Introduce HuggingFaceAPIGenerator. This text-generation component supports different Hugging Face APIs:

  • free Serverless Inference API
  • paid Inference Endpoints
  • self-hosted Text Generation Inference.

  This generator will replace the HuggingFaceTGIGenerator in the future.

• Introduce HuggingFaceAPITextEmbedder. This component can be used to embed strings using different Hugging Face APIs: - free Serverless Inference API - paid Inference Endpoints - self-hosted Text Embeddings Inference. This embedder will replace the HuggingFaceTEITextEmbedder in the future.

• Adds 'streaming_callback' parameter to 'HuggingFaceLocalGenerator', allowing users to handle streaming responses.

• Added a new EvaluationRunResult dataclass that wraps the results of an evaluation pipeline, allowing for its transformation and visualization.

• Add a new LLMEvaluator component that leverages LLMs through the OpenAI api to evaluate pipelines.

• Add DocumentRecallEvaluator, a Component that can be used to calculate the Recall single-hit or multi-hit metric given a list of questions, a list of expected documents for each question and the list of predicted documents for each question.

• Add SASEvaluator, it can be used to calculate Semantic Answer Similarity of generated answers from an LLM

• Introduce a new SparseEmbedding class which can be used to store a sparse vector representation of a Document. It will be instrumental to support Sparse Embedding Retrieval with the subsequent introduction of Sparse Embedders and Sparse Embedding Retrievers.

• Add a Zero Shot Text Router that uses an NLI model from HF to classify texts based on a set of provided labels and routes them based on the label they were classified with.

⚡️ Enhancement Notes

• add dimensions parameter to Azure OpenAI Embedders (AzureOpenAITextEmbedder and AzureOpenAIDocumentEmbedder) to fully support new embedding models like text-embedding-3-small, text-embedding-3-large and upcoming ones

• Now the DocumentSplitter adds the "page_number" field to the metadata of all output documents to keep track of the page of the original document it belongs to.

• Provides users the ability to customize text extraction from PDF files. It is particularly useful for PDFs with unusual layouts, such as those containing multiple text columns. For instance, users can configure the object to retain the reading order.

• Enhanced PromptBuilder to specify and enforce required variables in prompt templates.

• Set max_new_tokens default to 512 in Hugging Face generators.

• Enhanced the AzureOCRDocumentConverter to include advanced handling of tables and text. Features such as extracting preceding and following context for tables, merging multiple column headers, and enabling single column page layout for text have been introduced. This update furthers the flexibility and accuracy of document conversion within complex layouts.

• Enhanced DynamicChatPromptBuilder's capabilities by allowing all user and system messages to be templated with provided variables. This update ensures a more versatile and dynamic templating process, making chat prompt generation more efficient and customized to user needs.

• Improved HTML content extraction by attempting to use multiple extractors in order of priority until successful. An additional try_others parameter in HTMLToDocument, which is true by default, determines whether subsequent extractors are used after a failure. This enhancement decreases extraction failures, ensuring more dependable content retrieval.

• Enhanced FileTypeRouter with Regex Pattern Support for MIME Types: This introduces a significant enhancement to the FileTypeRouter, now featuring support for regex pattern matching for MIME types. This powerful addition allows for more granular control and flexibility in routing files based on their MIME types, enabling the handling of broad categories or specific MIME type patterns with ease. This feature is particularly beneficial for applications requiring sophisticated file classification and routing logic.

  Usage example: `python from haystack.components.routers import FileTypeRouter router = FileTypeRouter(mime_types=[r"text/.*", r"application/(pdf|json)"]) # Example files to classify file_paths = [ Path("document.pdf"), Path("report.json"), Path("notes.txt"), Path("image.png"), ] result = router.run(sources=file_paths) for mime_type, files in result.items(): print(f"MIME Type: {mime_type}, Files: {[str(file) for file in files]}")`

• Improved pipeline run tracing to include pipeline input/output data.

• In Jupyter notebooks, the image of the Pipeline will no longer be displayed automatically. The textual representation of the Pipeline will be displayed.

  To display the Pipeline image, use the show method of the Pipeline object.

• Add support for callbacks during pipeline deserialization. Currently supports a pre-init hook for components that can be used to inspect and modify the initialization parameters before the invocation of the component's __init__ method.

• pipeline.run accepts a set of component names whose intermediate outputs are returned in the final pipeline output dictionary.

• Pipeline.inputs and Pipeline.outputs can optionally include components input/output sockets that are connected.

• Refactor PyPDFToDocument to simplify support for custom PDF converters. PDF converters are classes that implement the PyPDFConverter protocol and have 3 methods: convert, to_dict and from_dict. The DefaultConverter class is provided as a default implementation.

• Add an __eq__ method to SparseEmbedding class to compare two SparseEmbedding objects.

⚠️ Deprecation Notes

• Deprecate HuggingFaceTGIChatGenerator. This component will be removed in Haystack 2.3.0. Use HuggingFaceAPIChatGenerator instead.
• Deprecate HuggingFaceTEIDocumentEmbedder. This component will be removed in Haystack 2.3.0. Use HuggingFaceAPIDocumentEmbedder instead.
• Deprecate HuggingFaceTGIGenerator. This component will be removed in Haystack 2.3.0. Use HuggingFaceAPIGenerator instead.
• Deprecate HuggingFaceTEITextEmbedder. This component will be removed in Haystack 2.3.0. Use HuggingFaceAPITextEmbedder instead.
• Using the converter_name parameter in the PyPDFToDocument component is deprecated. It will be removed in the 2.3.0 release. Use the converter parameter instead.

🐛 Bug Fixes

• Forward declaration of AnalyzeResult type in AzureOCRDocumentConverter.

  AnalyzeResult is already imported in a lazy import block. The forward declaration avoids issues when azure-ai-formrecognizer>=3.2.0b2 is not installed.

• The test_comparison_in test case in the base document store tests used to always pass, no matter how the in filtering logic was implemented in document stores. With the fix, the in logic is actually tested. Some tests might start to fail for document stores that don't implement the in filter correctly.

• Remove the usage of reserved keywords in the logger calls, causing a KeyError when setting the log level to DEBUG.

• Fixed a bug in the `MetaFieldRanker`: when the weight parameter was set to 0 in the run method, the component was incorrectly using the default weight parameter set in the __init__ method.

• Fixes Pipeline.run() logic so Components that have all their inputs with a default are run in the correct order. This happened we gather a list of Components to run internally when running the Pipeline in the order they are added during creation of the Pipeline. This caused some Components to run before they received all their inputs.

• Fix a bug when running a Pipeline that would cause it to get stuck in an infinite loop

• Fixes HuggingFaceTEITextEmbedder returning an embedding of incorrect shape when used with a Text-Embedding-Inference endpoint deployed using Docker.

• Add the @component decorator to HuggingFaceTGIChatGenerator. The lack of this decorator made it impossible to use the HuggingFaceTGIChatGenerator in a pipeline.

• Updated the SearchApiWebSearch component with new search format and allowed users to specify the search engine via the engine parameter in search_params. The default search engine is Google, making it easier for users to tailor their web searches.

• Fixed a bug in the `MetaFieldRanker`: when the ranking_mode parameter was overridden in the run method, the component was incorrectly using the ranking_mode parameter set in the __init__ method.

v2.1.0-rc0

⬆️ Upgrade Notes

• Removed the deprecated GPTGenerator and GPTChatGenerator components. Use OpenAIGenerator and OpenAIChatGeneratornotes instead.

• Update secret handling for the ExtractiveReader component using the Secret type.

  The default init parameter token is now required to either use a token or the environment HF_API_TOKEN variable if authentication is required - The on-disk local token file is no longer supported.

🚀 New Features

• Add a new pipeline template PredefinedPipeline.CHAT_WITH_WEBSITE to quickly create a pipeline that will answer questions based on data collected from one or more web pages.

  Usage example: `python from haystack import Pipeline, PredefinedPipeline pipe = Pipeline.from_template(PredefinedPipeline.CHAT_WITH_WEBSITE) result = pipe.run({ "fetcher": {"urls": ["https://haystack.deepset.ai/overview/quick-start"]}, "prompt": {"query": "How should I install Haystack?"}} ) print(result["llm"]["replies"][0])`

• Added option to instrument pipeline and component runs. This allows users to observe their pipeline runs and component runs in real-time via their chosen observability tool. Out-of-the-box support for OpenTelemetry and Datadog will be added in separate contributions.

  Example usage for [OpenTelemetry](https://opentelemetry.io/docs/languages/python/):

  1. Install OpenTelemetry SDK and exporter:
  `bash pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http`

  2. Configure OpenTelemetry SDK with your tracing provider and exporter:
  ```python from opentelemetry.sdk.resources import SERVICE_NAME, Resource

  from opentelemetry import trace from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor

  # Service name is required for most backends resource = Resource(attributes={ SERVICE_NAME: "haystack" })

  traceProvider = TracerProvider(resource=resource) processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces")) traceProvider.add_span_processor(processor) trace.set_tracer_provider(traceProvider)

  tracer = traceProvider.get_tracer("my_application")

  3. Create tracer
  `python import contextlib from typing import Optional, Dict, Any, Iterator from opentelemetry import trace from opentelemetry.trace import NonRecordingSpan from haystack.tracing import Tracer, Span from haystack.tracing import utils as tracing_utils import opentelemetry.trace class OpenTelemetrySpan(Span): def __init__(self, span: opentelemetry.trace.Span) -> None: self._span = span def set_tag(self, key: str, value: Any) -> None: coerced_value = tracing_utils.coerce_tag_value(value) self._span.set_attribute(key, coerced_value) class OpenTelemetryTracer(Tracer): def __init__(self, tracer: opentelemetry.trace.Tracer) -> None: self._tracer = tracer @contextlib.contextmanager def trace(self, operation_name: str, tags: Optional[Dict[str, Any]] = None) -> Iterator[Span]: with self._tracer.start_as_current_span(operation_name) as span: span = OpenTelemetrySpan(span) if tags: span.set_tags(tags) yield span def current_span(self) -> Optional[Span]: current_span = trace.get_current_span() if isinstance(current_span, NonRecordingSpan): return None return OpenTelemetrySpan(current_span)`

  4. Use the tracer with Haystack:
  `python from haystack import tracing haystack_tracer = OpenTelemetryTracer(tracer) tracing.enable_tracing(haystack_tracer)`

  1. Run your pipeline

• Enhanced OpenAPI integration by handling complex types of requests and responses in OpenAPIServiceConnector and OpenAPIServiceToFunctions.

• Added out-of-the-box support for the Datadog Tracer. This allows you to instrument pipeline and component runs using Datadog and send traces to your preferred backend.

  To use the Datadog Tracer you need to have the ddtrace package installed in your environment. To instruct Haystack to use the Datadog tracer, you have multiple options:

  • Run your Haystack application using the ddtrace command line tool as described in the the [ddtrace documentation](https://ddtrace.readthedocs.io/en/stable/installation_quickstart.html#tracing). This behavior can be disabled by setting the HAYSTACK_AUTO_TRACE_ENABLED_ENV_VAR environment variable to false.
  • Configure the tracer manually in your code using the ddtrace package: `python from haystack.tracing import DatadogTracer import haystack.tracing import ddtrace tracer = ddtrace.tracer tracing.enable_tracing(DatadogTracer(tracer))`

• Add AnswerExactMatchEvaluator, a component that can be used to calculate the Exact Match metric comparing a list of expected answers with a list of predicted answers.

• Added out-of-the-box support for the OpenTelemetry Tracer. This allows you to instrument pipeline and component runs using OpenTelemetry and send traces to your preferred backend.

  To use the OpenTelemetry Tracer you need to have the opentelemetry-sdk package installed in your environment. To instruct Haystack to use the OpenTelemetry Tracer, you have multiple options:

  * Run your Haystack application using the opentelemetry-instrument command line tool as described in the
  [OpenTelemetry documentation](https://opentelemetry.io/docs/languages/python/automatic/#configuring-the-agent).
  This behavior can be disabled by setting the HAYSTACK_AUTO_TRACE_ENABLED_ENV_VAR environment variable to false.

  * Configure the tracer manually in your code using the opentelemetry package:
  `python from opentelemetry import trace from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor # Service name is required for most backends resource = Resource(attributes={ SERVICE_NAME: "haystack" }) traceProvider = TracerProvider(resource=resource) processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces")) traceProvider.add_span_processor(processor) trace.set_tracer_provider(traceProvider) # Auto-configuration import haystack.tracing haystack.tracing.auto_enable_tracing() # Or explicitly from haystack.tracing import OpenTelemetryTracer tracer = traceProvider.get_tracer("my_application") tracing.enable_tracing(OpenTelemetryTracer(tracer))`

• Haystack now supports structured logging out-of-the box. Logging can be separated into 3 categories:

  • If [structlog](https://www.structlog.org/en/stable/) is not installed, Haystack will use the standard Python logging library with whatever configuration is present.
  • If structlog is installed, Haystack will log through [structlog](https://www.structlog.org/en/stable/) using structlog's console renderer. To disable structlog, set the environment variable HAYSTACK_LOGGING_IGNORE_STRUCTLOG_ENV_VAR to true.
  • To log in JSON, install [structlog](https://www.structlog.org/en/stable/) and
    • set the environment variable HAYSTACK_LOGGING_JSON to true or
    • enable JSON logging from Python `python import haystack.logging haystack.logging.configure_logging(use_json=True)`

⚡️ Enhancement Notes

• Allow code instrumentation to also trace the input and output of components. This is useful for debugging and understanding the behavior of components. This behavior is disabled by default and can be enabled with one of the following methods:

  • Set the environment variable HAYSTACK_CONTENT_TRACING_ENABLED_ENV_VAR to true before importing Haystack.

  • Enable content tracing in the code:
    `python from haystack import tracing tracing.tracer.is_content_tracing_enabled = True`

• Update Component protocol to fix type checking issues with some Language Servers. Most Language Servers and some type checkers would show warnings when calling Pipeline.add_component() as technically most `Component`s weren't respecting the protocol we defined.

• Added a new Logger implementation which eases and enforces logging via key-word arguments. This is an internal change only. The behavior of instances created via logging.getLogger is not affected.

• If using JSON logging in conjunction with tracing, Haystack will automatically add correlation IDs to the logs. This is done by getting the necessary information from the current span and adding it to the log record. You can customize this by overriding the get_correlation_data_for_logs of your tracer's span:

  `python from haystack.tracing import Span class OpenTelemetrySpan(Span): ... def get_correlation_data_for_logs(self) -> Dict[str, Any]: span_context = ... return {"trace_id": span_context.trace_id, "span_id": span_context.span_id}`

• The logging module now detects if the standard output is a TTY. If it is not and structlog is installed, it will automatically disable the console renderer and log in JSON format. This behavior can be overridden by setting the environment variable HAYSTACK_LOGGING_USE_JSON to false.

• Enhanced OpenAPI service connector to better handle method invocation with support for security schemes, refined handling of method arguments including URL/query parameters and request body, alongside improved error validation for method calls. This update enables more versatile interactions with OpenAPI services, ensuring compatibility with a wide range of API specifications.

Security Notes

• Remove the text value from a warning log in the TextLanguageRouter to avoid logging sensitive information. The text can be still be shown by switching to the debug log level.

  `python import logging logging.basicConfig(format="%(levelname)s - %(name)s - %(message)s", level=logging.WARNING) logging.getLogger("haystack").setLevel(logging.DEBUG)`

🐛 Bug Fixes

• Fix a bug in the MetaFieldRanker where the weight parameter passed to the run method was not being used.
• Pin the typing-extensions package to versions >= 4.7 to avoid [incompatibilities with the openai package](https://community.openai.com/t/error-while-importing-openai-from-open-import-openai/578166/26).
• Restore transparent background for images generated with Pipeline.draw and Pipeline.show
• Fix telemetry code that could cause a ValueError by trying to serialize a pipeline. Telemetry code does not serialize pipelines anymore.
• Fix Pipeline.run() mistakenly running a Component before it should. This can happen when a greedy variadic Component must be executed before a Component with default inputs.