DeepsetAmazonBedrockChatGenerator

Generate chat responses using Amazon Bedrock models with Haystack Platform integration.

Key Features

• Provides access to foundation models on Amazon Bedrock through Haystack Platform's Bedrock account.
• Supports models including Claude, Llama, and Titan from Amazon Bedrock.
• Accepts a list of ChatMessage objects for multi-turn conversations.
• Supports streaming responses and tool use (function calling).
• Passes custom generation parameters to the underlying model.

Configuration

1. Drag the DeepsetAmazonBedrockChatGenerator component onto the canvas from the Component Library.
2. Click the component to open the configuration panel.
3. On the General tab:
   1. Enter the model identifier, for example anthropic.claude-3-sonnet-20240229-v1:0.
4. Go to the Advanced tab to configure AWS credentials, generation parameters, and tools.

Connections

DeepsetAmazonBedrockChatGenerator accepts a list of ChatMessage objects (messages) as input.

Connect ChatPromptBuilder to the messages input to provide formatted chat prompts. Connect the replies output to AnswerBuilder to build the final response.

Usage Example

This is an example RAG pipeline using DeepsetAmazonBedrockChatGenerator with Amazon Bedrock models:

components:
  bm25_retriever:
    type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          index: 'default'
          max_chunk_bytes: 104857600
          embedding_dim: 768
          return_embedding: false
          method:
          mappings:
          settings:
          create_index: true
          http_auth:
          use_ssl:
          verify_certs:
          timeout:
      top_k: 20
      fuzziness: 0

  query_embedder:
    type: deepset_cloud_custom_nodes.embedders.nvidia.text_embedder.DeepsetNvidiaTextEmbedder
    init_parameters:
      normalize_embeddings: true
      model: intfloat/e5-base-v2

  embedding_retriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          index: 'default'
          max_chunk_bytes: 104857600
          embedding_dim: 768
          return_embedding: false
          method:
          mappings:
          settings:
          create_index: true
          http_auth:
          use_ssl:
          verify_certs:
          timeout:
      top_k: 20

  document_joiner:
    type: haystack.components.joiners.document_joiner.DocumentJoiner
    init_parameters:
      join_mode: concatenate

  ranker:
    type: deepset_cloud_custom_nodes.rankers.nvidia.ranker.DeepsetNvidiaRanker
    init_parameters:
      model: intfloat/simlm-msmarco-reranker
      top_k: 8

  meta_field_grouping_ranker:
    type: haystack.components.rankers.meta_field_grouping_ranker.MetaFieldGroupingRanker
    init_parameters:
      group_by: file_id
      subgroup_by:
      sort_docs_by: split_id

  answer_builder:
    type: deepset_cloud_custom_nodes.augmenters.deepset_answer_builder.DeepsetAnswerBuilder
    init_parameters:
      reference_pattern: acm

  ChatPromptBuilder:
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
    init_parameters:
      template:
      - _role: system
        _content:
        - text: "You are a helpful assistant answering the user's questions based on the provided documents. Do not use your own knowledge."
      - _role: user
        _content:
        - text: "Provided documents:\n{% for document in documents %}\nDocument [{{ loop.index }}]:\n{{ document.content }}\n{% endfor %}\n\nQuestion: {{ query }}\nAnswer:"
      variables:
      required_variables:

  generator:
    type: deepset_cloud_custom_nodes.generators.chat.deepset_amazon_bedrock_chat_generator.DeepsetAmazonBedrockChatGenerator
    init_parameters:
      model: anthropic.claude-3-sonnet-20240229-v1:0
      aws_region_name:
        type: env_var
        env_vars:
        - AWS_DEFAULT_REGION
        strict: false
      generation_kwargs:
        max_tokens: 1000
        temperature: 0.7
      streaming_callback:
      boto3_config:
      tools:

connections:
- sender: bm25_retriever.documents
  receiver: document_joiner.documents
- sender: query_embedder.embedding
  receiver: embedding_retriever.query_embedding
- sender: embedding_retriever.documents
  receiver: document_joiner.documents
- sender: document_joiner.documents
  receiver: ranker.documents
- sender: ranker.documents
  receiver: meta_field_grouping_ranker.documents
- sender: meta_field_grouping_ranker.documents
  receiver: answer_builder.documents
- sender: meta_field_grouping_ranker.documents
  receiver: ChatPromptBuilder.documents
- sender: ChatPromptBuilder.prompt
  receiver: generator.messages
- sender: generator.replies
  receiver: answer_builder.replies

inputs:
  query:
  - "bm25_retriever.query"
  - "query_embedder.text"
  - "ranker.query"
  - "answer_builder.query"
  - "ChatPromptBuilder.query"
  filters:
  - "bm25_retriever.filters"
  - "embedding_retriever.filters"

outputs:
  documents: "meta_field_grouping_ranker.documents"
  answers: "answer_builder.answers"

max_runs_per_component: 100

metadata: {}

Parameters

Inputs

Parameter	Type	Default	Description
messages	List[ChatMessage]		A list of chat messages to send to the model.

Outputs

Parameter	Type	Default	Description
replies	List[ChatMessage]		Generated chat message responses from the model.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
model	str		The Amazon Bedrock model identifier to use for generation.
aws_region_name	Optional[str]	None	AWS region name where the Bedrock service is accessed.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional parameters for generation, such as max_tokens, temperature, top_p.
streaming_callback	Optional[Callable]	None	A callback function called when a new token is received during streaming.
boto3_config	Optional[Dict[str, Any]]	None	Configuration for the boto3 client.
tools	Optional[List[Tool]]	None	A list of tools the model can use for function calling.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
messages	List[ChatMessage]		A list of chat messages to send to the model.