AnthropicChatGenerator

Use Anthropic's chat completion models to generate responses from a list of chat messages.

For a list of Anthropic models you can use, see Anthropic Models.

Key Features

• Supports Anthropic's Claude family of chat completion models.
• Accepts a list of ChatMessage objects as input, making it compatible with ChatPromptBuilder.
• Supports tool calling for agentic workflows.
• Supports streaming responses token by token.
• Customizable generation via generation_kwargs — any parameter that works with anthropic.Message.create also works here.

Configuration

1. Drag the AnthropicChatGenerator component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   1. Enter the model name in the Model field (for example, claude-sonnet-4-20250514).
   2. Make sure Haystack Platform is connected to Anthropic. You'll need an Anthropic API key. For details, see Use Anthropic Models.
4. Go to the Advanced tab to configure additional settings such as system_prompt, generation_kwargs, streaming_callback, timeout, and max_retries.

Connections

AnthropicChatGenerator receives a list of ChatMessage objects from ChatPromptBuilder through its messages input. It outputs a list of ChatMessage objects through its replies output. You typically connect its replies output to an OutputAdapter that converts the replies into a format that AnswerBuilder or DeepsetAnswerBuilder can accept.

Source Code

To check this component's source code, open chat_generator.py in the Haystack Core Integrations repository.

Usage Examples

Basic Configuration

  anthropic_chat_generator:
    type: haystack_integrations.components.generators.anthropic.chat.chat_generator.AnthropicChatGenerator
    init_parameters:
      model: claude-sonnet-4-20250514
      generation_kwargs:
        temperature: 0.7
        max_tokens: 500

This is a RAG pipeline that uses Claude Sonnet 4:

components:
  bm25_retriever:
    type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
            - ${OPENSEARCH_HOST}
          http_auth:
            - ${OPENSEARCH_USER}
            - ${OPENSEARCH_PASSWORD}
          use_ssl: true
          verify_certs: false
      top_k: 20

  query_embedder:
    type: haystack.components.embedders.sentence_transformers_text_embedder.SentenceTransformersTextEmbedder
    init_parameters:
      model: intfloat/e5-base-v2

  embedding_retriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
            - ${OPENSEARCH_HOST}
          http_auth:
            - ${OPENSEARCH_USER}
            - ${OPENSEARCH_PASSWORD}
          use_ssl: true
          verify_certs: false
      top_k: 20

  document_joiner:
    type: haystack.components.joiners.document_joiner.DocumentJoiner
    init_parameters:
      join_mode: concatenate

  ranker:
    type: haystack.components.rankers.transformers_similarity.TransformersSimilarityRanker
    init_parameters:
      model: intfloat/simlm-msmarco-reranker
      top_k: 8

  chat_prompt_builder:
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
    init_parameters:
      template:
        - _content:
            - text: |
                You are a helpful assistant answering questions based on the provided documents.
                If the documents don't contain the answer, say so.
                Do not use your own knowledge.
          _role: system
        - _content:
            - text: |
                Documents:
                {% for document in documents %}
                Document [{{ loop.index }}]:
                {{ document.content }}
                {% endfor %}

                Question: {{ query }}
          _role: user

  anthropic_chat_generator:
    type: haystack_integrations.components.generators.anthropic.chat.chat_generator.AnthropicChatGenerator
    init_parameters:
      model: claude-sonnet-4-20250514
      generation_kwargs:
        temperature: 0.7
        max_tokens: 500

  output_adapter:
    type: haystack.components.converters.output_adapter.OutputAdapter
    init_parameters:
      template: '{{ replies[0] }}'
      output_type: List[str]

  answer_builder:
    type: haystack.components.builders.answer_builder.AnswerBuilder
    init_parameters: {}

connections:
  - sender: bm25_retriever.documents
    receiver: document_joiner.documents
  - sender: query_embedder.embedding
    receiver: embedding_retriever.query_embedding
  - sender: embedding_retriever.documents
    receiver: document_joiner.documents
  - sender: document_joiner.documents
    receiver: ranker.documents
  - sender: ranker.documents
    receiver: chat_prompt_builder.documents
  - sender: ranker.documents
    receiver: answer_builder.documents
  - sender: chat_prompt_builder.prompt
    receiver: anthropic_chat_generator.messages
  - sender: anthropic_chat_generator.replies
    receiver: output_adapter.replies
  - sender: output_adapter.output
    receiver: answer_builder.replies

max_runs_per_component: 100

inputs:
  query:
    - bm25_retriever.query
    - query_embedder.text
    - ranker.query
    - chat_prompt_builder.query
    - answer_builder.query
  filters:
    - bm25_retriever.filters
    - embedding_retriever.filters

outputs:
  documents: ranker.documents
  answers: answer_builder.answers

metadata: {}

Parameters

Inputs

Parameter	Type	Description
messages	List[ChatMessage]	A list of ChatMessage objects representing the input messages.
generation_kwargs	Optional[Dict[str, Any]]	Additional keyword arguments for the model.
streaming_callback	Optional[StreamingCallbackT]	An optional callback function to handle streaming chunks.
tools	Optional[Union[List[Tool], Toolset]]	A list of tool objects or a toolset that the model can use. Each tool must have a unique name.

Outputs

Parameter	Type	Description
replies	List[ChatMessage]	A list of generated replies.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
api_key	Secret	Secret.from_env_var('ANTHROPIC_API_KEY')	The Anthropic API key.
model	str	claude-sonnet-4-20250514	The name of the Anthropic model to use.
streaming_callback	Optional[Callable[[StreamingChunk], None]]	None	An optional callback function to handle streaming chunks.
system_prompt	Optional[str]	None	An optional system prompt to use for generation.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for generation.
timeout	Optional[float]	None	The timeout for the request.
max_retries	Optional[int]	None	The maximum number of retries if a request fails.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
messages	List[ChatMessage]		A list of ChatMessage objects representing the input messages.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for generation. For a complete list, see Anthropic API documentation.
streaming_callback	Optional[Callable[[StreamingChunk], None]]	None	An optional callback function to handle streaming chunks.
tools	Optional[Union[List[Tool], Toolset]]	None	A list of tool objects or a toolset that the model can use. Each tool must have a unique name.

Related Information

• Use Anthropic Models
• Add Integrations