Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Use Anthropic Models

Use Anthropic models through Anthropic API in your pipelines.

You can use Anthropic's embedding and ranking models as well as LLMs. For a full list of supported models, see Anthropic documentation.

Prerequisites​

You need an active Anthropic API key to use Anthropic's models.

Use Anthropic Models​

First, connect Haystack Enterprise Platform to Anthropic through the Integrations page. You can set up the connection for a single workspace or for the whole organization:

Add Workspace-Level Integration​

  1. Click your profile icon and choose Settings.
  2. Go to Workspace>Integrations.
  3. Find the provider you want to connect and click Connect next to them.
  4. Enter the API key and any other required details.
  5. Click Connect. You can use this integration in pipelines and indexes in the current workspace.

Add Organization-Level Integration​

  1. Click your profile icon and choose Settings.
  2. Go to Organization>Integrations.
  3. Find the provider you want to connect and click Connect next to them.
  4. Enter the API key and any other required details.
  5. Click Connect. You can use this integration in pipelines and indexes in all workspaces in the current organization.

Then, add a component that uses an Anthropic model to your pipeline. Here are the available components:

Legacy

The previously used AnthropicChatGenerator and AnthropicGenerator are now legacy components. Use the LLM component instead.

Click the component card to open the configuration panel and choose the model and other settings.

Control Thinking Display for Claude 5 Models​

For Claude 5 model families (including Claude 5 Fable, Claude 5, Claude 5.5 Sonnet, Claude 5.5 Opus, and Claude 5.5 Haiku), you can control how the model's extended thinking appears in the response with the thinking_display parameter, available on the Advanced tab of the component configuration panel. Choose summarized if you want to see how the model reached its answer, for example while you test your pipeline. Choose omitted if you need only the answer in the response. The default depends on the model you choose.

For Claude 5.5 Haiku, the Advanced tab also lets you set how much the model reasons. By default, thinking_type is adaptive, so the model decides how much to think. Set disabled to turn thinking off. Use output_config_effort to choose an effort level: low, medium, high, xhigh, or max. The default is medium. max_tokens defaults to 128,000, the model's maximum. Lower it to limit response length.

Use Anthropic Models on Azure​

To use Anthropic models on Azure, create a custom model definition and choose Anthropic on Azure as the model provider:

  1. Depending on the scope of the custom model definition:
  • For a workspace-wide custom model definition, go to Workspace>Integrations and click Configure on the Custom Models Card.
  • For an organization-wide custom model definition, go to Organization>Integrations and click Configure on the Custom Models Card.
  1. Click Add model.
  2. Enter the following details:
  • Model name: A descriptive name for your custom model
  • Provider: Choose Anthropic on Azure.
  1. Configure the YAML settings:
  • Chat Generator: Define the component configuration including the endpoint or resource, and model name.
  • Model Parameters: Specify model parameters, like temperature and max output tokens. You can choose a model to copy its default parameters and adjust them as needed.
  1. Save your changes.

The model is now available in the model list when you add an Anthropic component to your pipeline.