DeepsetAmazonBedrockChatGenerator

Generate chat responses using Amazon Bedrock models with Haystack Platform integration.

DeepsetAmazonBedrockChatGenerator generates chat responses using Amazon Bedrock models hosted on Haystack Platform's Bedrock account. This component provides access to various foundation models available through Amazon Bedrock, including Claude, Llama, and Titan models.

Key Features

• Accesses Amazon Bedrock models through Haystack Platform's account — no separate AWS account required.
• Supports chat completion interface with ChatMessage input and output.
• Supports tool calling for agentic workflows.
• Supports streaming responses token by token.
• Configurable generation parameters such as max_tokens, temperature, and top_p.

Configuration

1. Drag the DeepsetAmazonBedrockChatGenerator component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   1. Enter the Amazon Bedrock model identifier in the Model field (for example, anthropic.claude-3-sonnet-20240229-v1:0).
   2. Set the AWS region name where the Bedrock service is accessed.
4. Go to the Advanced tab to configure generation_kwargs, streaming_callback, boto3_config, and tools.

Connections

DeepsetAmazonBedrockChatGenerator receives a list of ChatMessage objects from ChatPromptBuilder through its messages input. It outputs a list of ChatMessage objects through its replies output, which you connect to AnswerBuilder or DeepsetAnswerBuilder.

Usage Examples

Basic Configuration

  generator:
    type: 
      deepset_cloud_custom_nodes.generators.chat.deepset_amazon_bedrock_chat_generator.DeepsetAmazonBedrockChatGenerator
    init_parameters:
      model: anthropic.claude-3-sonnet-20240229-v1:0
      aws_region_name:
        type: env_var
        env_vars:
        - AWS_DEFAULT_REGION
        strict: false
      generation_kwargs:
        max_tokens: 1000
        temperature: 0.7

This is an example RAG pipeline using DeepsetAmazonBedrockChatGenerator with Amazon Bedrock models:

components:
  bm25_retriever:
    type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          index: 'default'
          max_chunk_bytes: 104857600
          embedding_dim: 768
          return_embedding: false
          method:
          mappings:
          settings:
          create_index: true
          http_auth:
          use_ssl:
          verify_certs:
          timeout:
      top_k: 20
      fuzziness: 0

  query_embedder:
    type: deepset_cloud_custom_nodes.embedders.nvidia.text_embedder.DeepsetNvidiaTextEmbedder
    init_parameters:
      normalize_embeddings: true
      model: intfloat/e5-base-v2

  embedding_retriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          index: 'default'
          max_chunk_bytes: 104857600
          embedding_dim: 768
          return_embedding: false
          method:
          mappings:
          settings:
          create_index: true
          http_auth:
          use_ssl:
          verify_certs:
          timeout:
      top_k: 20

  document_joiner:
    type: haystack.components.joiners.document_joiner.DocumentJoiner
    init_parameters:
      join_mode: concatenate

  ranker:
    type: deepset_cloud_custom_nodes.rankers.nvidia.ranker.DeepsetNvidiaRanker
    init_parameters:
      model: intfloat/simlm-msmarco-reranker
      top_k: 8

  meta_field_grouping_ranker:
    type: haystack.components.rankers.meta_field_grouping_ranker.MetaFieldGroupingRanker
    init_parameters:
      group_by: file_id
      subgroup_by:
      sort_docs_by: split_id

  answer_builder:
    type: deepset_cloud_custom_nodes.augmenters.deepset_answer_builder.DeepsetAnswerBuilder
    init_parameters:
      reference_pattern: acm

  ChatPromptBuilder:
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
    init_parameters:
      template:
      - _role: system
        _content:
        - text: "You are a helpful assistant answering the user's questions based on the provided documents. Do not use your own knowledge."
      - _role: user
        _content:
        - text: "Provided documents:\n{% for document in documents %}\nDocument [{{ loop.index }}]:\n{{ document.content }}\n{% endfor %}\n\nQuestion: {{ query }}\nAnswer:"
      variables:
      required_variables:

  generator:
    type: deepset_cloud_custom_nodes.generators.chat.deepset_amazon_bedrock_chat_generator.DeepsetAmazonBedrockChatGenerator
    init_parameters:
      model: anthropic.claude-3-sonnet-20240229-v1:0
      aws_region_name:
        type: env_var
        env_vars:
        - AWS_DEFAULT_REGION
        strict: false
      generation_kwargs:
        max_tokens: 1000
        temperature: 0.7
      streaming_callback:
      boto3_config:
      tools:

connections:
- sender: bm25_retriever.documents
  receiver: document_joiner.documents
- sender: query_embedder.embedding
  receiver: embedding_retriever.query_embedding
- sender: embedding_retriever.documents
  receiver: document_joiner.documents
- sender: document_joiner.documents
  receiver: ranker.documents
- sender: ranker.documents
  receiver: meta_field_grouping_ranker.documents
- sender: meta_field_grouping_ranker.documents
  receiver: answer_builder.documents
- sender: meta_field_grouping_ranker.documents
  receiver: ChatPromptBuilder.documents
- sender: ChatPromptBuilder.prompt
  receiver: generator.messages
- sender: generator.replies
  receiver: answer_builder.replies

inputs:
  query:
  - "bm25_retriever.query"
  - "query_embedder.text"
  - "ranker.query"
  - "answer_builder.query"
  - "ChatPromptBuilder.query"
  filters:
  - "bm25_retriever.filters"
  - "embedding_retriever.filters"

outputs:
  documents: "meta_field_grouping_ranker.documents"
  answers: "answer_builder.answers"

max_runs_per_component: 100

metadata: {}

Parameters

Inputs

Parameter	Type	Description
messages	List[ChatMessage]	A list of chat messages to send to the model.

Outputs

Parameter	Type	Description
replies	List[ChatMessage]	Generated chat message responses from the model.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
model	str		The Amazon Bedrock model identifier to use for generation.
aws_region_name	Optional[str]	None	AWS region name where the Bedrock service is accessed.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional parameters for generation, such as max_tokens, temperature, top_p.
streaming_callback	Optional[Callable]	None	A callback function called when a new token is received during streaming.
boto3_config	Optional[Dict[str, Any]]	None	Configuration for the boto3 client.
tools	Optional[List[Tool]]	None	A list of tools the model can use for function calling.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
messages	List[ChatMessage]		A list of chat messages to send to the model.