github deepset-ai/haystack v1.18.0
on GitHub

latest releases: v2.5.1, v2.5.1-rc2, v2.5.1-rc1...
15 months ago

⭐️ Highlights

🗄️ Using LLMs Hosted on AWS SageMaker

We are thrilled to share that Haystack now supports the use of open source LLMs deployed to AWS SageMaker! This means you can easily host your models with AWS SageMaker and use them with PromptNode by simply providing the inference endpoint name and your aws_profile_name 👇 

from haystack.nodes import PromptNode

prompt_node = PromptNode(
    model_name_or_path="sagemaker-model-endpoint-name", 
    model_kwargs={"aws_profile_name": "my_aws_profile_name", 
	          "aws_region_name": "your-region-k"}
)

For more details on model deployment, check out the documentation.

🗂️ PromptHub

Exciting news! We’re introducing PromptHub: A place for ready-made prompts for the most common NLP tasks. The best part about it is that you can easily import prompts from the hub into Haystack. For example, if you want to use deepset/topic-classification prompt with Haystack, all you need is the prompt name, and that's it! 👇 

import os

from haystack.nodes import PromptNode, PromptTemplate

topic_classifier_template = PromptTemplate("deepset/topic-classification")
prompt_node = PromptNode(model_name_or_path="text-davinci-003", api_key=os.environ.get("OPENAI_API_KEY"))
prompt_node.prompt(prompt_template=topic_classifier_template, documents="YOUR_DOCUMENTS", options=["A LIST OF TOPICS"])

Check out the PromptHub and discover other prompt options for various NLP tasks.

🛠️ Adding Tools to ConversationalAgent

The wait is over – you can now greatly enhance your chat application by adding various tools using the tools parameter of ConversationalAgent. 

from haystack.agents import Tool
from haystack.agents.conversational import ConversationalAgent

search_tool = Tool(
    name="USA_Presidents_QA",
    pipeline_or_node=presidents_qa,
    description="useful for when you need to answer questions related to the presidents of the USA.",
)

conversational_agent = ConversationalAgent(prompt_node=prompt_node, tools=[search_tool])
conversational_agent.run("YOUR QUERY")

Go to our docs to learn more about various tools you can use.

⚠️ Breaking Changes

🔦 A new optional dependency: inference

To simplify the installation of Haystack, we removed some redundant dependencies (like PyTorch) for users who rely on LLMs through APIs. This means you will get a smaller package and faster Haystack installation if you want to use Haystack with OpenAI models or Transformers models via Hugging Face Inference API. If you prefer to run models locally, for example on your GPU, pip install farm-haystack[inference] will install all required dependencies as before.

🔁 PromptTemplate parameter changes

As of v1.18, PromptTemplate accepts only prompt and output_parser and doesn’t support name and prompt_text parameters anymore. See an example of how you can migrate to the new PromptTemplate 👇🏼 

qa_template = PromptTemplate(
    name="question-answering",
    promp_text="""
    Given the context please answer the question. Context: {join(documents)}; 
    Question: {query}; 
    Answer:
    """,
    output_parser=AnswerParser()
)
qa_template = PromptTemplate(
    prompt="""
    Given the context please answer the question. Context: {join(documents)}; 
    Question: {query}; 
    Answer:
    """,
    output_parser=AnswerParser()
)

🚫 Seq2SeqGenerator and RAGenerator deprecation

With Haystack 1.18, we removed the Seq2SeqGenerator and RAGenerator from Haystack. We recommend using the more powerful PromptNode instead. See the announcement for more details.

What's Changed

Breaking Changes

  • PromptHub integration in PromptNode by @ZanSara in #4879
  • chore: remove deprecated node PDFToTextOCRConverter by @masci in #4982
  • refactor: Use globally defined request timeout in ElasticsearchDocumentStore and OpenSearchDocumentStore by @bogdankostic in #5064
  • feat!: simplify weaviate auth by @hsm207 in #5115
  • feat!: Add extra for inference dependencies such as torch by @julian-risch in #5147
  • Remove deprecated param return_table_cell by @masci in #5218

Pipeline

DocumentStores

  • docs: updating docstrings to say OpenSearch and backlink to correct docs by @dtaivpp in #5000
  • feat: Add batching for querying in ElasticsearchDocumentStore and OpenSearchDocumentStore by @bogdankostic in #5063
  • feat: Add batch_size parameter and cast timeout_config value to tuple for WeaviateDocumentStore by @bogdankostic in #5079
  • fix: changing document scores by @benheckmann in #5090

Documentation

  • fix: Fix CohereInvocationLayer _ensure_token_limit not returning resized prompt by @silvanocerza in #4978
  • feat: Add prompt_template to conversational agent init params by @julian-risch in #4994
  • feat: Allow setting java options when launching Elasticsearch / OpenSearch by @bogdankostic in #5002
  • refactor: Adapt retriever benchmarks script by @bogdankostic in #5004
  • refactor: Add reader-retriever benchmark script by @bogdankostic in #5006
  • refactor: Adapt running benchmarks by @bogdankostic in #5007
  • fix: Move check for default PromptTemplates in PromptTemplate itself by @ZanSara in #5018
  • chore: Simplify DefaultPromptHandler logic and add tests by @silvanocerza in #4979
  • feat: prompts caching from PromptHub by @ZanSara in #5048
  • fix: Fix handling of streaming response in AnthropicClaudeInvocationLayer by @silvanocerza in #4993
  • feat: pass model parameters to HFLocalInvocationLayer via model_kwargs, enabling direct model usage by @vblagoje in #4956
  • feat: Consider prompt_node's default_prompt_template in agent by @julian-risch in #5095
  • fix: rename requests.py into requests_utils.py by @ZanSara in #5099
  • docs: update CLI readme by @dfokina in #5129
  • fix: small improvement to pipeline v2 tests by @ZanSara in #5153
  • feat: Optional Content Moderation for OpenAI PromptNode & OpenAIAnswerGenerator by @benheckmann in #5017
  • feat: Update ConversationalAgent by @bilgeyucel in #5065
  • fix: Check Agent's prompt template variables and prompt resolver parameters are aligned by @vblagoje in #5163
  • feat: add ensure token limit for direct prompting of ChatGPT by @sjrl in #5166
  • chore: remove deprecated Seq2SeqGenerator and RAGenerator by @anakin87 in #5180
  • fix: model_tokenizer in openai text completion tokenization details by @michaelfeil in #5104
  • chore: Remove add_tool from ToolsManager by @bogdankostic in #5192
  • feat: enable LLMs hosted via AWS SageMaker in PromptNode by @tholor in #5155
  • feat: Add CohereRanker node using Cohere reranking endpoint by @sjrl in #5152
  • Fix: FARMReader - Consider the max number of labels/answers during training by @anakin87 in #5197
  • fix: Support all SageMaker HF text generation models (other than Falcon) by @vblagoje in #5205

Other Changes

New Contributors

Full Changelog: v1.17.2...v1.18.0

Check out latest releases or
releases around deepset-ai/haystack v1.18.0

Don't miss a new haystack release

NewReleases is sending notifications on new releases.

Get notifications