DeepsetAmazonBedrockGenerator

Generate text using large language models hosted on Haystack Platform's Amazon Bedrock account, so you don't need to create your own account.

Deprecation Notice

This component is deprecated. It will continue to work in your existing pipelines for now. You can replace it with the AmazonBedrockChatGenerator component.

Key Features

• Uses models in Amazon Bedrock through Haystack Platform's Bedrock account — no personal AWS account required.
• Supports a range of models including Meta's Llama 2 and other Amazon Bedrock models.
• Accepts a text prompt and returns a list of generated replies.
• Supports streaming responses through a configurable callback function.
• Passes region and model-specific parameters for flexible configuration.

Configuration

1. Drag the DeepsetAmazonBedrockGenerator component onto the canvas from the Component Library.
2. Click the component to open the configuration panel.
3. On the General tab:
   1. Enter the model name, for example meta.llama2-13b-chat-v1.
4. Go to the Advanced tab to configure AWS credentials and generation parameters.

Connections

DeepsetAmazonBedrockGenerator accepts a prompt string and optional generation_kwargs as input.

Connect PromptBuilder to the prompt input to provide formatted instructions. Connect the replies output to AnswerBuilder to build GeneratedAnswer objects.

Usage Example

This example uses the Llama2 model hosted on Amazon Bedrock to generate answers. It gets the prompt with documents from PromptBuilder and then sends the generated replies to AnswerBuilder:

components:
  # ...
  prompt_builder:
      type: haystack.components.builders.prompt_builder.PromptBuilder
      init_parameters:
        template: |-
          You are a technical expert.
          You answer questions truthfully based on provided documents.
          For each document check whether it is related to the question.
          Only use documents that are related to the question to answer it.
          Ignore documents that are not related to the question.
          If the answer exists in several documents, summarize them.
          Only answer based on the documents provided. Don't make things up.
          If the documents can't answer the question or you are unsure say: 'The answer can't be found in the text'.
          These are the documents:
          {% for document in documents %}
          Document[{{ loop.index }}]:
          {{ document.content }}
          {% endfor %}
          Question: {{question}}
          Answer:

  llm:
      type: deepset_cloud_custom_nodes.generators.deepset_amazon_bedrock_generator.DeepsetAmazonBedrockGenerator
      init_parameters:
         model: "meta.llama2-13b-chat-v1"
         aws_region_name: us-east-1 # Region name is required
         max_length: 400
         model_max_length: 4096
         temperature: 0
         streaming_callback: deepset_cloud_custom_nodes.callbacks.streaming.streaming_callback # makes this generator stream

  answer_builder:
      type: haystack.components.builders.answer_builder.AnswerBuilder

connections:
  # ...
  - sender: prompt_builder.prompt
    receiver: llm.prompt
  - sender: llm.replies
    receiver: answer_builder.replies

Parameters

Inputs

Parameter	Type	Default	Description
prompt	str		The prompt with instructions for the model.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for text generation. These parameters potentially override the parameters passed in pipeline configuration. For details on supported parameters, check the model's documentation.

Outputs

Parameter	Type	Default	Description
replies	List[str]		A list of strings containing the generated responses.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
model	str		The name or path of the model in Amazon Bedrock.
aws_region_name	str	None	None
max_length	int	None	100
truncate	bool	None	True
streaming_callback	Callable[[StreamingChunk], None]	None	None
kwargs	Any		Additional keyword arguments for the model. These arguments are model-specific. For supported arguments, check the model's documentation.

Run Method Parameters

This component doesn't accept any runtime parameters.