AmazonBedrockChatGenerator

Generate chat responses using large language models hosted on Amazon Bedrock via the Bedrock Converse API. Bedrock provides access to foundation models from Anthropic, Meta, Mistral, Amazon, and others through a unified API.

Key Features

• Access to models from multiple providers through Amazon Bedrock's Converse API.
• Supports Claude, Llama, Mistral, Amazon Titan, and other foundation models.
• Supports global inference profiles for automatic cross-region routing (for example, global.anthropic.claude-sonnet-4-6).
• Supports streaming responses through a configurable callback.
• Supports tool calling for agentic workflows.
• Supports AWS Bedrock Guardrails for content safety.
• Supports prompt caching via tools_cachepoint_config and system_cachepoint_config.
• Flexible AWS authentication: access keys, session tokens, IAM roles, or named profiles.

Configuration

1. Drag the AmazonBedrockChatGenerator component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   1. Set the model to the Bedrock model you want to use (for example, global.anthropic.claude-sonnet-4-6 for Claude Sonnet via global inference, or amazon.nova-pro-v1:0 for Amazon Nova Pro). For available models, see the Amazon Bedrock documentation.
   2. Create secrets with your AWS credentials. For instructions, see Create Secrets. Use these environment variable names:
      • AWS_ACCESS_KEY_ID
      • AWS_SECRET_ACCESS_KEY
      • AWS_DEFAULT_REGION (required for region-specific models without global inference profiles)
4. Go to the Advanced tab to configure generation_kwargs such as maxTokens, temperature, and topP.

Connections

AmazonBedrockChatGenerator receives a list of ChatMessage objects as input, typically from PromptBuilder or ChatPromptBuilder. It outputs a list of reply ChatMessage objects you can connect to AnswerBuilder or other downstream components.

Source Code

To check this component's source code, open chat_generator.py in the Haystack Core Integrations repository.

Usage Examples

Basic Configuration

  AmazonBedrockChatGenerator:
    type: haystack_integrations.components.generators.amazon_bedrock.chat.chat_generator.AmazonBedrockChatGenerator
    init_parameters:
      model: global.anthropic.claude-sonnet-4-6
      aws_access_key_id:
        type: env_var
        env_vars:
        - AWS_ACCESS_KEY_ID
        strict: false
      aws_secret_access_key:
        type: env_var
        env_vars:
        - AWS_SECRET_ACCESS_KEY
        strict: false
      generation_kwargs:
        maxTokens: 1024
        temperature: 0.7

Using the Component in a Pipeline

# haystack-pipeline
components:
  prompt_builder:
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
    init_parameters:
      required_variables: "*"
      template:
      - role: system
        content: You are a helpful assistant. Answer questions based on the provided documents.
      - role: user
        content: |
          Documents:
          {% for doc in documents %}
          {{ doc.content }}
          {% endfor %}
          Question: {{ question }}

  llm:
    type: haystack_integrations.components.generators.amazon_bedrock.chat.chat_generator.AmazonBedrockChatGenerator
    init_parameters:
      model: global.anthropic.claude-sonnet-4-6
      aws_access_key_id:
        type: env_var
        env_vars:
        - AWS_ACCESS_KEY_ID
        strict: false
      aws_secret_access_key:
        type: env_var
        env_vars:
        - AWS_SECRET_ACCESS_KEY
        strict: false
      generation_kwargs:
        maxTokens: 1024

  answer_builder:
    type: deepset_cloud_custom_nodes.augmenters.deepset_answer_builder.DeepsetAnswerBuilder
    init_parameters:
      reference_pattern: acm

connections:
- sender: prompt_builder.prompt
  receiver: llm.messages
- sender: llm.replies
  receiver: answer_builder.replies

max_runs_per_component: 100

metadata: {}

inputs:
  query:
  - answer_builder.query
  - prompt_builder.question
  documents:
  - prompt_builder.documents

outputs:
  answers: answer_builder.answers

Parameters

Inputs

Parameter	Type	Description
messages	List[ChatMessage]	A list of chat messages representing the conversation so far.

Outputs

Parameter	Type	Description
replies	List[ChatMessage]	A list of generated reply messages from the model.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
model	str		The Amazon Bedrock model ID or cross-region inference profile (for example, global.anthropic.claude-sonnet-4-6). For a full list, see the Amazon Bedrock documentation.
aws_access_key_id	Optional[Secret]	Secret.from_env_var("AWS_ACCESS_KEY_ID", strict=False)	The AWS access key ID.
aws_secret_access_key	Optional[Secret]	Secret.from_env_var("AWS_SECRET_ACCESS_KEY", strict=False)	The AWS secret access key.
aws_session_token	Optional[Secret]	Secret.from_env_var("AWS_SESSION_TOKEN", strict=False)	The AWS session token for temporary credentials.
aws_region_name	Optional[Secret or str]	Secret.from_env_var("AWS_DEFAULT_REGION", strict=False)	The AWS region. Required for region-specific models; not needed for global inference profiles.
aws_profile_name	Optional[Secret]	Secret.from_env_var("AWS_PROFILE", strict=False)	The AWS named profile to use for authentication.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional generation parameters passed to Bedrock, such as maxTokens, temperature, topP, and stopSequences.
streaming_callback	Optional[Callable]	None	A callback function for streaming responses. When set, the model streams tokens as they're generated.
boto3_config	Optional[Dict[str, Any]]	None	Additional configuration for the boto3 client (for example, timeouts).
tools	Optional[List[Tool]]	None	A list of tools the model can use.
guardrail_config	Optional[Dict[str, str]]	None	Configuration for Amazon Bedrock Guardrails content filtering.
tools_cachepoint_config	Optional[Dict[str, str]]	None	Prompt caching configuration for the tools block.
system_cachepoint_config	Optional[Dict[str, str]]	None	Prompt caching configuration for the system prompt block.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
messages	List[ChatMessage]		A list of chat messages representing the conversation.
streaming_callback	Optional[Callable]	None	A callback function to override the init-time streaming callback.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional generation parameters to override init-time values.
tools	Optional[List[Tool]]	None	Tools to make available to the model, overriding init-time tools.

Related Information

• AmazonBedrockTextEmbedder
• AmazonBedrockDocumentEmbedder
• Pipeline Components