OpenAIResponsesChatGenerator

Generate text with LLM models using OpenAI's Responses API with support for reasoning models.

Basic Information

• Type: haystack.components.generators.chat.openai_responses.OpenAIResponsesChatGenerator
• Components it can connect with:
  • ChatPromptBuilder: Sends rendered chat prompts to OpenAIResponsesChatGenerator.
  • DeepsetAnswerBuilder: Receives generated replies from OpenAIResponsesChatGenerator through OutputAdapter.
  • OutputAdapter: Converts chat messages to the format needed by downstream components.

Inputs

Parameter	Type	Description
messages	List[ChatMessage]	A list of ChatMessage objects representing the input messages.
streaming_callback	Optional[StreamingCallbackT]	A callback function called when a new token is received from the stream.
generation_kwargs	Optional[Dict[str, Any]]	Additional keyword arguments for text generation. These parameters override the parameters in pipeline configuration. For supported parameters, see OpenAI documentation.
tools	Optional[Union[List[Tool], Toolset, List[dict]]]	A list of tools or OpenAI/MCP tool definitions for which the model can prepare calls. If set, it overrides the tools parameter set during component initialization. Can accept either a list of Tool objects, or OpenAI/MCP tool definitions as dictionaries. You cannot pass OpenAI/MCP tools and Haystack tools together.
tools_strict	Optional[bool]	Whether to enable strict schema adherence for tool calls. If set to True, the model follows the schema exactly, but this may increase latency. If set, it overrides the tools_strict parameter in pipeline configuration.

Outputs

Parameter	Type	Description
replies	List[ChatMessage]	A list containing the generated responses as ChatMessage instances.

Overview

OpenAIResponsesChatGenerator uses OpenAI's Responses API to generate chat completions. It supports gpt-4, gpt-5, and o-series models (reasoning models like o1, o3-mini). The default model is gpt-5-mini.

The Responses API is designed for models that can reason. It supports features like reasoning summaries, multi-turn conversations with previous response IDs, and structured outputs.

You can customize text generation by passing parameters to the OpenAI API. Use the generation_kwargs argument when you initialize the component or when you run it. Any parameter that works with openai.Responses.create will work here too.

For a list of supported OpenAI API parameters, see OpenAI documentation.

Authorization

You need an OpenAI API key to use this component. Connect deepset to your OpenAI account on the Integrations page. For details, see Use OpenAI Models.

Reasoning Support

One of the key features of the Responses API is support for reasoning models. You can configure reasoning behavior using the reasoning parameter in generation_kwargs.

The reasoning parameter accepts:

• effort: Specifies the level of reasoning effort for the mode. Possible values are:"low", "medium", or "high".
• summary: Specifies how to generate reasoning summaries. You can choose: "auto" or "generate_summary": True/False

info

OpenAI does not return the actual reasoning tokens, but you can view the summary, if enabled. For more details, see the OpenAI Reasoning documentation.

Multi-Turn Conversations

The Responses API supports multi-turn conversations using previous_response_id. You can pass the response ID from a previous turn to maintain conversation context.

Structured Output

OpenAIResponsesChatGenerator supports structured output generation through the text_format and text parameters in generation_kwargs:

• text_format: Pass a Pydantic model to define the structure.
• text: Pass a JSON schema directly.

Model Compatibility and Limitations

• Both Pydantic models and JSON schemas are supported for latest models starting from GPT-4o.
• If both text_format and text are provided, text_format takes precedence and the JSON schema passed to text is ignored.
• Streaming is not supported when using structured outputs.
• Older models only support basic JSON mode through {"type": "json_object"}. For details, see OpenAI JSON mode documentation.
• For complete information, check the OpenAI Structured Outputs documentation.

Usage Example

This is an example RAG pipeline with OpenAIResponsesChatGenerator and DeepsetAnswerBuilder connected through OutputAdapter:

components:
  bm25_retriever:
    type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
            - ${OPENSEARCH_HOST}
          http_auth:
            - ${OPENSEARCH_USER}
            - ${OPENSEARCH_PASSWORD}
          use_ssl: true
          verify_certs: false
      top_k: 20

  query_embedder:
    type: haystack.components.embedders.sentence_transformers_text_embedder.SentenceTransformersTextEmbedder
    init_parameters:
      model: intfloat/e5-base-v2

  embedding_retriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
            - ${OPENSEARCH_HOST}
          http_auth:
            - ${OPENSEARCH_USER}
            - ${OPENSEARCH_PASSWORD}
          use_ssl: true
          verify_certs: false
      top_k: 20

  document_joiner:
    type: haystack.components.joiners.document_joiner.DocumentJoiner
    init_parameters:
      join_mode: concatenate

  ranker:
    type: haystack.components.rankers.transformers_similarity.TransformersSimilarityRanker
    init_parameters:
      model: intfloat/simlm-msmarco-reranker
      top_k: 8

  chat_prompt_builder:
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
    init_parameters:
      template:
        - _content:
            - text: "You are a helpful assistant answering questions based on the provided documents.\nIf the documents don't contain the answer, say so.\nDo not use your own knowledge.\n"
          _role: system
        - _content:
            - text: "Documents:\n{% for document in documents %}\nDocument [{{ loop.index }}]:\n{{ document.content }}\n{% endfor %}\n\nQuestion: {{ query }}\n"
          _role: user

  openai_responses_chat_generator:
    type: haystack.components.generators.chat.openai_responses.OpenAIResponsesChatGenerator
    init_parameters:
      model: gpt-5-mini
      generation_kwargs:
        reasoning:
          effort: low
          summary: auto
        temperature: 0.7
        max_tokens: 500

  output_adapter:
    type: haystack.components.converters.output_adapter.OutputAdapter
    init_parameters:
      template: '{{ replies[0] }}'
      output_type: List[str]

  answer_builder:
    type: haystack.components.builders.answer_builder.AnswerBuilder
    init_parameters: {}

connections:
  - sender: bm25_retriever.documents
    receiver: document_joiner.documents
  - sender: query_embedder.embedding
    receiver: embedding_retriever.query_embedding
  - sender: embedding_retriever.documents
    receiver: document_joiner.documents
  - sender: document_joiner.documents
    receiver: ranker.documents
  - sender: ranker.documents
    receiver: chat_prompt_builder.documents
  - sender: ranker.documents
    receiver: answer_builder.documents
  - sender: chat_prompt_builder.prompt
    receiver: openai_responses_chat_generator.messages
  - sender: openai_responses_chat_generator.replies
    receiver: output_adapter.replies
  - sender: output_adapter.output
    receiver: answer_builder.replies

max_runs_per_component: 100

inputs:
  query:
    - bm25_retriever.query
    - query_embedder.text
    - ranker.query
    - chat_prompt_builder.query
    - answer_builder.query
  filters:
    - bm25_retriever.filters
    - embedding_retriever.filters

outputs:
  documents: ranker.documents
  answers: answer_builder.answers

metadata: {}

Parameters

Init parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
api_key	Secret	Secret.from_env_var('OPENAI_API_KEY')	The OpenAI API key. Set it on the Integrations page.
model	str	gpt-5-mini	The name of the model to use.
streaming_callback	Optional[StreamingCallbackT]	None	A callback function called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument.
api_base_url	Optional[str]	None	An optional base URL.
organization	Optional[str]	None	Your organization ID. See production best practices.
generation_kwargs	Optional[Dict[str, Any]]	None	Other parameters to use for the model, sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some supported parameters: temperature (sampling temperature, higher values mean more risks), top_p (nucleus sampling probability mass), previous_response_id (ID of the previous response for multi-turn conversations), text_format (Pydantic model for structured outputs), text (JSON schema for structured outputs), reasoning (dictionary with effort and summary parameters for reasoning models).
timeout	Optional[float]	30.0	Timeout for OpenAI client calls. If not set, it defaults to the OPENAI_TIMEOUT environment variable or 30 seconds.
max_retries	Optional[int]	five	Maximum number of retries to contact OpenAI after an internal error. If not set, it defaults to the OPENAI_MAX_RETRIES environment variable or five.
tools	Optional[Union[List[Tool], Toolset, List[dict]]]	None	A list of tools, a Toolset, or OpenAI/MCP tool definitions for which the model can prepare calls. This parameter can accept either a list of Tool objects, a Toolset instance, or OpenAI/MCP tool definitions as dictionaries. Note: You cannot pass OpenAI/MCP tools and Haystack tools together.
tools_strict	boolean	False	Whether to enable strict schema adherence for tool calls. If set to True, the model follows exactly the schema provided in the parameters field of the tool definition, but this may increase latency. In Response API, tool calls are strict by default.
http_client_kwargs	Optional[Dict[str, Any]]	None	A dictionary of keyword arguments to configure a custom httpx.Client or httpx.AsyncClient. For more information, see the HTTPX documentation.

Run method parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
messages	List[ChatMessage]		A list of ChatMessage instances representing the input messages.
streaming_callback	Optional[StreamingCallbackT]	None	A callback function called when a new token is received from the stream.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for text generation. These parameters override the parameters in pipeline configuration. For supported parameters, see OpenAI documentation.
tools	Optional[Union[List[Tool], Toolset, List[dict]]]	None	A list of tools, a Toolset, or OpenAI/MCP tool definitions for which the model can prepare calls. If set, it overrides the tools parameter in pipeline configuration. Can accept either a list of Tool objects, a Toolset instance, or OpenAI/MCP tool definitions as dictionaries. Note: You cannot pass OpenAI/MCP tools and Haystack tools together.
tools_strict	Optional[bool]	None	Whether to enable strict schema adherence for tool calls. If set to True, the model follows exactly the schema provided in the parameters field of the tool definition, but this may increase latency. If set, it overrides the tools_strict parameter in pipeline configuration.