OpenAIChatGenerator

Complete chats using OpenAI's large language models (LLMs).

Key Features

• Works with GPT-4, GPT-5, and o-series models via the OpenAI API.
• Accepts ChatMessage objects as input and returns generated replies as ChatMessage objects.
• Supports streaming responses via a callback function.
• Supports tool calls for agentic workflows.
• Supports customizable generation parameters via generation_kwargs.

Configuration

1. Drag the OpenAIChatGenerator component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   • Set the model name.
   • Set the OpenAI API key. Connect the platform to your OpenAI account on the Integrations page first. For details, see Use OpenAI Models.
4. Go to the Advanced tab to configure api_base_url, timeout, max_retries, generation_kwargs, and http_client_kwargs.

Connections

OpenAIChatGenerator accepts a list of ChatMessage objects as input. Connect its messages input to the prompt output of ChatPromptBuilder.

It outputs replies as a list of ChatMessage objects. Connect its replies output through OutputAdapter to DeepsetAnswerBuilder or AnswerBuilder.

Source Code

To check this component's source code, open openai.py in the Haystack repository.

Usage Examples

Basic Configuration

  openai_chat_generator:
    type: haystack.components.generators.chat.openai.OpenAIChatGenerator
    init_parameters:
      model: gpt-5-mini
      generation_kwargs:
        temperature: 0.7
        max_tokens: 500

This is an example RAG pipeline with OpenAIChatGenerator and DeepsetAnswerBuilder connected through OutputAdapter:

components:
  bm25_retriever:
    type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
            - ${OPENSEARCH_HOST}
          http_auth:
            - ${OPENSEARCH_USER}
            - ${OPENSEARCH_PASSWORD}
          use_ssl: true
          verify_certs: false
      top_k: 20

  query_embedder:
    type: haystack.components.embedders.sentence_transformers_text_embedder.SentenceTransformersTextEmbedder
    init_parameters:
      model: intfloat/e5-base-v2

  embedding_retriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
            - ${OPENSEARCH_HOST}
          http_auth:
            - ${OPENSEARCH_USER}
            - ${OPENSEARCH_PASSWORD}
          use_ssl: true
          verify_certs: false
      top_k: 20

  document_joiner:
    type: haystack.components.joiners.document_joiner.DocumentJoiner
    init_parameters:
      join_mode: concatenate

  ranker:
    type: haystack.components.rankers.transformers_similarity.TransformersSimilarityRanker
    init_parameters:
      model: intfloat/simlm-msmarco-reranker
      top_k: 8

  chat_prompt_builder:
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
    init_parameters:
      template:
        - _content:
            - text: "You are a helpful assistant answering questions based on the provided documents.\nIf the documents don't contain the answer, say so.\nDo not use your own knowledge.\n"
          _role: system
        - _content:
            - text: "Documents:\n{% for document in documents %}\nDocument [{{ loop.index }}]:\n{{ document.content }}\n{% endfor %}\n\nQuestion: {{ query }}\n"
          _role: user

  openai_chat_generator:
    type: haystack.components.generators.chat.openai.OpenAIChatGenerator
    init_parameters:
      model: gpt-5-mini
      generation_kwargs:
        temperature: 0.7
        max_tokens: 500

  output_adapter:
    type: haystack.components.converters.output_adapter.OutputAdapter
    init_parameters:
      template: '{{ replies[0] }}'
      output_type: List[str]

  answer_builder:
    type: haystack.components.builders.answer_builder.AnswerBuilder
    init_parameters: {}

connections:
  - sender: bm25_retriever.documents
    receiver: document_joiner.documents
  - sender: query_embedder.embedding
    receiver: embedding_retriever.query_embedding
  - sender: embedding_retriever.documents
    receiver: document_joiner.documents
  - sender: document_joiner.documents
    receiver: ranker.documents
  - sender: ranker.documents
    receiver: chat_prompt_builder.documents
  - sender: ranker.documents
    receiver: answer_builder.documents
  - sender: chat_prompt_builder.prompt
    receiver: openai_chat_generator.messages
  - sender: openai_chat_generator.replies
    receiver: output_adapter.replies
  - sender: output_adapter.output
    receiver: answer_builder.replies

max_runs_per_component: 100

inputs:
  query:
    - bm25_retriever.query
    - query_embedder.text
    - ranker.query
    - chat_prompt_builder.query
    - answer_builder.query
  filters:
    - bm25_retriever.filters
    - embedding_retriever.filters

outputs:
  documents: ranker.documents
  answers: answer_builder.answers

metadata: {}

Parameters

Inputs

Parameter	Type	Description
messages	List[ChatMessage]	A list of ChatMessage instances representing the input messages.
streaming_callback	Optional[StreamingCallbackT]	A callback function called when a new token is received from the stream.
generation_kwargs	Optional[Dict[str, Any]]	Additional keyword arguments for text generation. These parameters override the parameters in pipeline configuration. For supported parameters, see OpenAI documentation.
tools	Optional[Union[List[Tool], Toolset]]	A list of tools or a Toolset for which the model can prepare calls. If set, it overrides the tools parameter set during component initialization.
tools_strict	Optional[bool]	Whether to enable strict schema adherence for tool calls. If set, it overrides the tools_strict parameter in pipeline configuration.

Outputs

Parameter	Type	Description
replies	List[ChatMessage]	A list containing the generated responses as ChatMessage instances.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
api_key	Secret	Secret.from_env_var('OPENAI_API_KEY')	The OpenAI API key. Set it on the Integrations page.
model	str	gpt-5-mini	The name of the model to use.
streaming_callback	Optional[StreamingCallbackT]	None	A callback function called when a new token is received from the stream.
api_base_url	Optional[str]	None	An optional base URL.
organization	Optional[str]	None	Your organization ID. See production best practices.
generation_kwargs	Optional[Dict[str, Any]]	None	Other parameters to use for the model, sent directly to the OpenAI endpoint. See OpenAI documentation for more details.
timeout	Optional[float]	30.0	Timeout for OpenAI client calls. If not set, it defaults to the OPENAI_TIMEOUT environment variable or 30 seconds.
max_retries	Optional[int]	five	Maximum number of retries to contact OpenAI after an internal error. If not set, it defaults to the OPENAI_MAX_RETRIES environment variable or five.
tools	Optional[Union[List[Tool], Toolset]]	None	A list of tools or a Toolset for which the model can prepare calls.
tools_strict	boolean	False	Whether to enable strict schema adherence for tool calls. If set to True, the model follows exactly the schema provided in the parameters field of the tool definition, but this may increase latency.
http_client_kwargs	Optional[Dict[str, Any]]	None	A dictionary of keyword arguments to configure a custom httpx.Client or httpx.AsyncClient. For more information, see the HTTPX documentation.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
messages	List[ChatMessage]		A list of ChatMessage instances representing the input messages.
streaming_callback	Optional[StreamingCallbackT]	None	A callback function called when a new token is received from the stream.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for text generation. These parameters override the parameters in pipeline configuration. For supported parameters, see OpenAI documentation.
tools	Optional[Union[List[Tool], Toolset]]	None	A list of tools or a Toolset for which the model can prepare calls. If set, it overrides the tools parameter in pipeline configuration.
tools_strict	Optional[bool]	None	Whether to enable strict schema adherence for tool calls. If set, it overrides the tools_strict parameter in pipeline configuration.

Related Information

• Use OpenAI Models