Prerequisites

• An Amazon Web Services account with an Amazon SageMaker Studio domain configured. If you need help, see the Amazon SageMaker documentation.
• A draft pipeline with a PromptNode. For help, see Create Pipeline and Generative Question Answering Pipelines.

Using a Model in Your Pipeline

There are two steps you must perform:

1. Deploy the model in SageMaker.
2. Add the model to your deepset Cloud pipeline.

Deploying the Model in SageMaker

1. Log in to AWS and open Amazon SageMaker.
2. In the navigation, click Studio.
3. Once you're in Studio, select your user profile in the Get Started section and click Open Studio.
   
4. From the navigation, choose SageMaker JumpStart>Models, notebooks, solutions.
5. In the Foundation Models: Text Generation section, open the model you want to use and click Deploy.
6. Copy the endpoint name from the Endpoint details section.

Adding the Model to Your Pipeline

1. Log in to deepset Cloud and go to Pipelines.
2. Click the three dots next to the pipeline that should use the model hosted on SageMaker and click Edit.
3. In the YAML components section, find PromptNode and specify the following parameters for it:
   YAML

   components:
     - name: PromptNode 
       type: PromptNode
       params:
         default_prompt_template: question-answering #type the name of the prompt you want to use
         model_name_or_path:  <sagemaker_endpoint_name>
         model_kwargs: 
           aws_profile_name: <your_aws_profile_name>
           aws_region_name: <your_aws_region>
   

4. Save your pipeline.

What's Next

Now you can deploy your pipeline to experiment with different prompts, gather feedback, and eventually use it for search in production.