Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Use Anthropic Models

Use Anthropic models through Anthropic API in your pipelines.

You can use Anthropic's embedding and ranking models as well as LLMs. For a full list of supported models, see Anthropic documentation.

Prerequisites​

You need an active Anthropic API key to use Anthropic's models.

Use Anthropic Models​

First, connect Haystack Enterprise Platform to Anthropic through the Integrations page. You can set up the connection for a single workspace or for the whole organization:

Add Workspace-Level Integration​

  1. Click your profile icon and choose Settings.
  2. Go to Workspace>Integrations.
  3. Find the provider you want to connect and click Connect next to them.
  4. Enter the API key and any other required details.
  5. Click Connect. You can use this integration in pipelines and indexes in the current workspace.

Add Organization-Level Integration​

  1. Click your profile icon and choose Settings.
  2. Go to Organization>Integrations.
  3. Find the provider you want to connect and click Connect next to them.
  4. Enter the API key and any other required details.
  5. Click Connect. You can use this integration in pipelines and indexes in all workspaces in the current organization.

Then, add a component that uses an Anthropic model to your pipeline. Here are the available components:

Legacy

The previously used AnthropicChatGenerator and AnthropicGenerator are now legacy components. Use the LLM component instead.

Click the component card to open the configuration panel and choose the model and other settings.

Use Anthropic Models on Azure​

To use Anthropic models on Azure, create a custom model definition and choose Anthropic on Azure as the model provider:

  1. Depending on the scope of the custom model definition:
  • For a workspace-wide custom model definition, go to Workspace>Integrations and click Configure on the Custom Models Card.
  • For an organization-wide custom model definition, go to Organization>Integrations and click Configure on the Custom Models Card.
  1. Click Add model.
  2. Enter the following details:
  • Model name: A descriptive name for your custom model
  • Provider: Choose Anthropic on Azure.
  1. Configure the YAML settings:
  • Chat Generator: Define the component configuration including the endpoint or resource, and model name.
  • Model Parameters: Specify model parameters, like temperature and max output tokens. You can choose a model to copy its default parameters and adjust them as needed.
  1. Save your changes.

The model is now available in the model list when you add an Anthropic component to your pipeline.