Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

OpenAIResponsesChatGenerator

Generate text with LLM models using OpenAI's Responses API with support for reasoning models. The default model is gpt-5-mini.

The Responses API is designed for models that can reason. It supports features like reasoning summaries, multi-turn conversations with previous response IDs, and structured outputs.

Key Features​

  • Uses OpenAI's Responses API, optimized for reasoning models (gpt-4, gpt-5, o-series).
  • Supports reasoning configuration with effort and summary parameters.
  • Supports multi-turn conversations using previous_response_id.
  • Supports structured output via Pydantic models (text_format) or JSON schema (text).
  • Supports tool calling with Haystack tools, Toolsets, and OpenAI/MCP tool definitions.
  • Streaming support for token-by-token responses.

Configuration​

  1. Drag the OpenAIResponsesChatGenerator component onto the canvas from the Component Library.
  2. Click on the component to open the configuration panel.
  3. On the General tab:
    • Make sure Haystack Platform is connected to your OpenAI account. For help, see Use OpenAI Models.
    • Select the model to use.
  4. Go to the Advanced tab to configure additional settings such as generation_kwargs, timeout, max_retries, http_client_kwargs, and tool options.

Connections​

OpenAIResponsesChatGenerator receives rendered chat prompts from ChatPromptBuilder through its messages input. It sends generated replies through its replies output to downstream components such as OutputAdapter or DeepsetAnswerBuilder.

Source Code​

To check this component's source code, open openai_responses.py in the Haystack repository.

Usage Examples​

Basic Configuration​

  openai_responses_chat_generator:
    type: haystack.components.generators.chat.openai_responses.OpenAIResponsesChatGenerator
    init_parameters:
      model: gpt-5-mini
      generation_kwargs:
        reasoning:
          effort: low
          summary: auto
        temperature: 0.7
        max_tokens: 500

This is an example RAG pipeline with OpenAIResponsesChatGenerator and DeepsetAnswerBuilder connected through OutputAdapter:

components:
  bm25_retriever:
    type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
            - ${OPENSEARCH_HOST}
          http_auth:
            - ${OPENSEARCH_USER}
            - ${OPENSEARCH_PASSWORD}
          use_ssl: true
          verify_certs: false
      top_k: 20

  query_embedder:
    type: haystack.components.embedders.sentence_transformers_text_embedder.SentenceTransformersTextEmbedder
    init_parameters:
      model: intfloat/e5-base-v2

  embedding_retriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
            - ${OPENSEARCH_HOST}
          http_auth:
            - ${OPENSEARCH_USER}
            - ${OPENSEARCH_PASSWORD}
          use_ssl: true
          verify_certs: false
      top_k: 20

  document_joiner:
    type: haystack.components.joiners.document_joiner.DocumentJoiner
    init_parameters:
      join_mode: concatenate

  ranker:
    type: haystack.components.rankers.transformers_similarity.TransformersSimilarityRanker
    init_parameters:
      model: intfloat/simlm-msmarco-reranker
      top_k: 8

  chat_prompt_builder:
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
    init_parameters:
      template:
        - _content:
            - text: "You are a helpful assistant answering questions based on the provided documents.\nIf the documents don't contain the answer, say so.\nDo not use your own knowledge.\n"
          _role: system
        - _content:
            - text: "Documents:\n{% for document in documents %}\nDocument [{{ loop.index }}]:\n{{ document.content }}\n{% endfor %}\n\nQuestion: {{ query }}\n"
          _role: user

  openai_responses_chat_generator:
    type: haystack.components.generators.chat.openai_responses.OpenAIResponsesChatGenerator
    init_parameters:
      model: gpt-5-mini
      generation_kwargs:
        reasoning:
          effort: low
          summary: auto
        temperature: 0.7
        max_tokens: 500

  output_adapter:
    type: haystack.components.converters.output_adapter.OutputAdapter
    init_parameters:
      template: '{{ replies[0] }}'
      output_type: List[str]

  answer_builder:
    type: haystack.components.builders.answer_builder.AnswerBuilder
    init_parameters: {}

connections:
  - sender: bm25_retriever.documents
    receiver: document_joiner.documents
  - sender: query_embedder.embedding
    receiver: embedding_retriever.query_embedding
  - sender: embedding_retriever.documents
    receiver: document_joiner.documents
  - sender: document_joiner.documents
    receiver: ranker.documents
  - sender: ranker.documents
    receiver: chat_prompt_builder.documents
  - sender: ranker.documents
    receiver: answer_builder.documents
  - sender: chat_prompt_builder.prompt
    receiver: openai_responses_chat_generator.messages
  - sender: openai_responses_chat_generator.replies
    receiver: output_adapter.replies
  - sender: output_adapter.output
    receiver: answer_builder.replies

max_runs_per_component: 100

inputs:
  query:
    - bm25_retriever.query
    - query_embedder.text
    - ranker.query
    - chat_prompt_builder.query
    - answer_builder.query
  filters:
    - bm25_retriever.filters
    - embedding_retriever.filters

outputs:
  documents: ranker.documents
  answers: answer_builder.answers

metadata: {}

Parameters​

Inputs​

ParameterTypeDescription
messagesList[ChatMessage]A list of ChatMessage objects representing the input messages.
streaming_callbackOptional[StreamingCallbackT]A callback function called when a new token is received from the stream.
generation_kwargsOptional[Dict[str, Any]]Additional keyword arguments for text generation. These parameters override the parameters in pipeline configuration. For supported parameters, see OpenAI documentation.
toolsOptional[Union[List[Tool], Toolset, List[dict]]]A list of tools or OpenAI/MCP tool definitions for which the model can prepare calls. If set, it overrides the tools parameter set during component initialization. Can accept either a list of Tool objects, or OpenAI/MCP tool definitions as dictionaries. You cannot pass OpenAI/MCP tools and Haystack tools together.
tools_strictOptional[bool]Whether to enable strict schema adherence for tool calls. If set to True, the model follows the schema exactly, but this may increase latency. If set, it overrides the tools_strict parameter in pipeline configuration.

Outputs​

ParameterTypeDescription
repliesList[ChatMessage]A list containing the generated responses as ChatMessage instances.

Init Parameters​

These are the parameters you can configure in Pipeline Builder:

ParameterTypeDefaultDescription
api_keySecretSecret.from_env_var('OPENAI_API_KEY')The OpenAI API key. Set it on the Integrations page.
modelstrgpt-5-miniThe name of the model to use.
streaming_callbackOptional[StreamingCallbackT]NoneA callback function called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument.
api_base_urlOptional[str]NoneAn optional base URL.
organizationOptional[str]NoneYour organization ID. See production best practices.
generation_kwargsOptional[Dict[str, Any]]NoneOther parameters to use for the model, sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some supported parameters: temperature (sampling temperature, higher values mean more risks), top_p (nucleus sampling probability mass), previous_response_id (ID of the previous response for multi-turn conversations), text_format (Pydantic model for structured outputs), text (JSON schema for structured outputs), reasoning (dictionary with effort and summary parameters for reasoning models).
timeoutOptional[float]30.0Timeout for OpenAI client calls. If not set, it defaults to the OPENAI_TIMEOUT environment variable or 30 seconds.
max_retriesOptional[int]fiveMaximum number of retries to contact OpenAI after an internal error. If not set, it defaults to the OPENAI_MAX_RETRIES environment variable or five.
toolsOptional[Union[List[Tool], Toolset, List[dict]]]NoneA list of tools, a Toolset, or OpenAI/MCP tool definitions for which the model can prepare calls. This parameter can accept either a list of Tool objects, a Toolset instance, or OpenAI/MCP tool definitions as dictionaries. Note: You cannot pass OpenAI/MCP tools and Haystack tools together.
tools_strictbooleanFalseWhether to enable strict schema adherence for tool calls. If set to True, the model follows exactly the schema provided in the parameters field of the tool definition, but this may increase latency. In Response API, tool calls are strict by default.
http_client_kwargsOptional[Dict[str, Any]]NoneA dictionary of keyword arguments to configure a custom httpx.Client or httpx.AsyncClient. For more information, see the HTTPX documentation.

Run Method Parameters​

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

ParameterTypeDefaultDescription
messagesList[ChatMessage]A list of ChatMessage instances representing the input messages.
streaming_callbackOptional[StreamingCallbackT]NoneA callback function called when a new token is received from the stream.
generation_kwargsOptional[Dict[str, Any]]NoneAdditional keyword arguments for text generation. These parameters override the parameters in pipeline configuration. For supported parameters, see OpenAI documentation.
toolsOptional[Union[List[Tool], Toolset, List[dict]]]NoneA list of tools, a Toolset, or OpenAI/MCP tool definitions for which the model can prepare calls. If set, it overrides the tools parameter in pipeline configuration. Can accept either a list of Tool objects, a Toolset instance, or OpenAI/MCP tool definitions as dictionaries. Note: You cannot pass OpenAI/MCP tools and Haystack tools together.
tools_strictOptional[bool]NoneWhether to enable strict schema adherence for tool calls. If set to True, the model follows exactly the schema provided in the parameters field of the tool definition, but this may increase latency. If set, it overrides the tools_strict parameter in pipeline configuration.