DeepsetAmazonBedrockGenerator

Generate text using large language models hosted on deepset's Amazon Bedrock account, so you don't need to create your own account.

Suggest Edits

Basic Information

Pipeline type: Query
Type: deepset_cloud_custom_nodes.generators.deepset_amazon_bedrock_generator.DeepsetAmazonBedrockGenerator
Components it can connect with:
- PromptBuilder: DeepsetAmazonBedrockGenerator receives the prompt from PromptBuilder.
- AnswerBuilder: DeepsetAmazonBedrockGenerator sends the generated replies to AnswerBuilder, which uses them to build GeneratedAnswer objects.

Inputs

Required Inputs

Name	Type	Description
`prompt`	String	The prompt with instructions for the model.

Optional Inputs

Name	Type	Default	Description
`generation_kwargs`	Dictionary of string and any	`None`	Additional keyword arguments you want to pass to the generator.

Outputs

Name	Type	Description
`replies`	List of strings	Generated responses.

Overview

DeepsetAmazonBedrockGenerator makes it possible to use models in Amazon Bedrock through deepset's Bedrock account. You don't need your own Bedrock account to use these models. To use models through your own Bedrock account, use AmazonBedrockGenerator.

For a full list of models, see Amazon Bedrock documentation.

Authentication

DeepsetAmazonBedrockGenerator connects to deepset's Bedrock account without requiring you to pass any credentials. You can use models hosted in Bedrock right away.

Usage Example

This example uses the Llama2 model hosted on Amazon Bedrock to generate answers. It gets the prompt with documents from PromptBuilder and then sends the generated replies to AnswerBuilder:

components:
...
  prompt_builder:
      type: haystack.components.builders.prompt_builder.PromptBuilder
      init_parameters:
        template: |-
          You are a technical expert.
          You answer questions truthfully based on provided documents.
          For each document check whether it is related to the question.
          Only use documents that are related to the question to answer it.
          Ignore documents that are not related to the question.
          If the answer exists in several documents, summarize them.
          Only answer based on the documents provided. Don't make things up.
          If the documents can't answer the question or you are unsure say: 'The answer can't be found in the text'.
          These are the documents:
          {% for document in documents %}
          Document[{{ loop.index }}]:
          {{ document.content }}
          {% endfor %}
          Question: {{question}}
          Answer:

    llm:
      type: deepset_cloud_custom_nodes.generators.deepset_amazon_bedrock_generator.DeepsetAmazonBedrockGenerator
      init_parameters:
         model: "meta.llama2-13b-chat-v1"
         aws_region_name: us-east-1 # Region name is required
         max_length: 400
         model_max_length: 4096
         temperature: 0
       
    answer_builder:
       type: haystack.components.builders.answer_builder.AnswerBuilder
      ...

connections:
...
 - sender: prompt_builder.prompt
   receiver: llm.prompt
 - sender: llm.replies
   receiver: answer_builder.replies
   ...

Init Parameters

Parameter	Type	Possible values	Description
`model`	String	Default: `None`	The ID of the model to use. For model IDs, check Amazon Bedrock documentation. Required.
`aws_region_name`	String	Default: `None`	The AWS region. Make sure it's supported by Bedrock. Required.
`kwargs`	Dictionary of strings		Additional keyword arguments to be passed to the model. Optional.

Updated 6 months ago