Use Amazon Bedrock Models

You can use models hosted in your own Bedrock account.

---

You can use embedding models and LLMs hosted on Amazon Bedrock through Bedrock's API. You can connect to your Bedrock account using your Bedrock API key of IAM access key. For a full list of supported models, see Amazon Bedrock documentation.

Prerequisites

• To connect using your Bedrock API key, you must have a valid API key.
• To connect using your IAM access key, you must have a valid access key ID and secret access key for your Amazon region.

For details, see Amazon Bedrock model access and Amazon Bedrock documentation.

Using Bedrock Models

First, connect Haystack Enterprise Platform to Amazon Bedrock by passing your Bedrock API key or IAM access key on the Integrations page. You can do so for a single workspace or for the whole organization:

Add Workspace-Level Integration

1. Click your profile icon and choose Settings.
2. Go to Workspace>Integrations.
3. Find the provider you want to connect and click Connect next to them.
4. Enter the API key and any other required details.
5. Click Connect. You can use this integration in pipelines and indexes in the current workspace.

Add Organization-Level Integration

1. Click your profile icon and choose Settings.
2. Go to Organization>Integrations.
3. Find the provider you want to connect and click Connect next to them.
4. Enter the API key and any other required details.
5. Click Connect. You can use this integration in pipelines and indexes in all workspaces in the current organization.

Then, add a component that uses the Bedrock model to your pipeline. Here is a list of components by the model type they use:

• Embedding models (used to calculate embeddings for text):
  • AmazonBedrockTextEmbedder: Calculates embeddings for text, such as query. Often used in query pipelines to embed query and then pass it to an embedding retriever.
  • AmazonBedrockDocumentEmbedder: Calculates embeddings for documents. Often used in indexes to embed documents and pass them to DocumentWriter.
    Embedding Models in Query Pipelines and Indexes

    The embedding model you use to embed documents in your indexing pipeline must be the same as the embedding model you use to embed the query in your query pipeline.

    This means the embedders for your indexing and query pipelines must match. For example, if you use CohereDocumentEmbedder to embed your documents, you should use CohereTextEmbedder with the same model to embed your queries.

• LLMs:
  • LLM: Generates text, often used in RAG pipelines.
• Rankers:
  • AmazonBedrockRanker: Ranks documents based on their similarity to the query using Amazon Bedrock models.

Legacy

The previously used AmazonBedrockGenerator, AmazonBedrockChatGenerator, DeepsetAmazonBedrockGenerator, and DeepsetAmazonBedrockChatGenerator are deprecated. Use the LLM component instead.

Usage Examples

This is an example of how to use embedding models and an LLM hosted on Bedrock in an index and a query pipeline (each in a separate tab):

Index

components:
  # ...
    splitter:
      type: haystack.components.preprocessors.document_splitter.DocumentSplitter
      init_parameters:
        split_by: word
        split_length: 250
        split_overlap: 30

    document_embedder:
      type: haystack_integrations.components.embedders.amazon_bedrock.document_embedder.AmazonBedrockDocumentEmbedder
      init_parameters:
        model: "cohere.embed-english-v3"

    writer:
      type: haystack.components.writers.document_writer.DocumentWriter
      init_parameters:
        document_store:
          type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
          init_parameters:
            embedding_dim: 768
            similarity: cosine
        policy: OVERWRITE
        
connections:  # Defines how the components are connected
  # ...
  - sender: splitter.documents
    receiver: document_embedder.documents
  - sender: document_embedder.documents
    receiver: writer.documents

Pipeline

# haystack-pipeline
inputs:
  query:
  - AmazonBedrockTextEmbedder.text
  filters: []
  files: []
  messages: []

components:
  LLM:
    type: haystack.components.generators.chat.llm.LLM
    init_parameters:
      chat_generator:
        init_parameters:
          model: global.anthropic.claude-sonnet-4-6
        type: haystack_integrations.components.generators.amazon_bedrock.chat.chat_generator.AmazonBedrockChatGenerator
      system_prompt:
      user_prompt: >-
        {% message role="user" %}

        You are a technical expert.

        You answer questions truthfully based on provided documents.

        For each document check whether it is related to the question.

        Only use documents that are related to the question to answer it.

        Ignore documents that are not related to the question.

        If the answer exists in several documents, summarize them.

        Only answer based on the documents provided. Don't make things up.

        If the documents can't answer the question or you are unsure say: 'The
        answer can't be found in the text'.

        These are the documents:

        {% for document in documents %}

        Document[{{ loop.index }}]:

        {{ document.content }}

        {% endfor %}

        Question: {{ question }}

        Answer:

        {% endmessage %}
      required_variables: "*"
      streaming_callback:

  AmazonBedrockTextEmbedder:
    type: haystack_integrations.components.embedders.amazon_bedrock.text_embedder.AmazonBedrockTextEmbedder
    init_parameters:
      model:
      aws_access_key_id:
        type: env_var
        env_vars:
        - AWS_ACCESS_KEY_ID
        strict: false
      aws_secret_access_key:
        type: env_var
        env_vars:
        - AWS_SECRET_ACCESS_KEY
        strict: false
      aws_session_token:
        type: env_var
        env_vars:
        - AWS_SESSION_TOKEN
        strict: false
      aws_region_name:
        type: env_var
        env_vars:
        - AWS_DEFAULT_REGION
        strict: false
      aws_profile_name:
        type: env_var
        env_vars:
        - AWS_PROFILE
        strict: false
      boto3_config:

  OpenSearchEmbeddingRetriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      filters:
      top_k: 10
      filter_policy: replace
      custom_query:
      raise_on_failure: true
      efficient_filtering: true
      search_kwargs:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          index: ""
          max_chunk_bytes: 104857600
          embedding_dim: 768
          return_embedding: false
          settings:
            index.knn: true
          create_index: true

outputs:
  answers:
  documents:
  messages: LLM.messages

connections:
- sender: AmazonBedrockTextEmbedder.embedding
  receiver: OpenSearchEmbeddingRetriever.query_embedding
- sender: OpenSearchEmbeddingRetriever.documents
  receiver: LLM.documents

Related Information

• Add Custom Model Definitions