OpenAIGenerator

Generate text using OpenAI's large language models (LLMs).

Key Features

• Works with GPT-4, GPT-5, and o-series models via the OpenAI API.
• Accepts string prompts and returns string replies.
• Supports streaming responses via a callback function.
• Supports customizable generation parameters via generation_kwargs.
• Supports optional system prompts to configure model behavior.

Configuration

1. Drag the OpenAIGenerator component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   • Set the model name.
   • Set the OpenAI API key. Connect the platform to your OpenAI account on the Integrations page first. For details, see Use OpenAI Models.
   • Optionally, set a system_prompt to configure model behavior.
4. Go to the Advanced tab to configure api_base_url, timeout, max_retries, generation_kwargs, and http_client_kwargs.

Connections

OpenAIGenerator accepts a prompt string as input. Connect its prompt input to the prompt output of PromptBuilder.

It outputs replies as a list of strings and meta as a list of metadata dictionaries. Connect its replies output to AnswerBuilder.

Source Code

To check this component's source code, open openai.py in the Haystack repository.

Usage Examples

Basic Configuration

  OpenAIGenerator:
    type: haystack.components.generators.openai.OpenAIGenerator
    init_parameters:
      api_key:
        type: env_var
        env_vars:
        - OPENAI_API_KEY
        strict: false
      model: gpt-5-mini

Here's an example RAG pipeline using OpenAIGenerator:

components:
  bm25_retriever:
    type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          - ${OPENSEARCH_HOST}
          index: ''
          max_chunk_bytes: 104857600
          embedding_dim: 768
          return_embedding: false
          method:
          mappings:
          settings:
          create_index: true
          http_auth:
          - ${OPENSEARCH_USER}
          - ${OPENSEARCH_PASSWORD}
          use_ssl: true
          verify_certs: false
          timeout:
      top_k: 10

  prompt_builder:
    type: haystack.components.builders.prompt_builder.PromptBuilder
    init_parameters:
      template: |-
        You are a helpful assistant.
        Answer the question based on the provided documents.
        If the documents don't contain the answer, say so.

        Documents:
        {% for document in documents %}
        {{ document.content }}
        {% endfor %}

        Question: {{question}}
        Answer:

  answer_builder:
    type: haystack.components.builders.answer_builder.AnswerBuilder
    init_parameters: {}
  OpenAIGenerator:
    type: haystack.components.generators.openai.OpenAIGenerator
    init_parameters:
      api_key:
        type: env_var
        env_vars:
        - OPENAI_API_KEY
        strict: false
      model: gpt-5-mini
      streaming_callback:
      api_base_url:
      organization:
      system_prompt:
      generation_kwargs:
      timeout:
      max_retries:
      http_client_kwargs:

connections:
- sender: bm25_retriever.documents
  receiver: prompt_builder.documents
- sender: bm25_retriever.documents
  receiver: answer_builder.documents
- sender: prompt_builder.prompt
  receiver: OpenAIGenerator.prompt
- sender: OpenAIGenerator.replies
  receiver: answer_builder.replies

max_runs_per_component: 100

inputs:
  query:
  - bm25_retriever.query
  - prompt_builder.question
  - answer_builder.query

outputs:
  answers: answer_builder.answers

metadata: {}

Parameters

Inputs

Parameter	Type	Description
prompt	str	The string prompt to use for text generation.
system_prompt	Optional[str]	The system prompt to use for text generation. If this runtime system prompt is omitted, the system prompt defined at initialization time is used.
streaming_callback	Optional[StreamingCallbackT]	A callback function called when a new token is received from the stream.
generation_kwargs	Optional[Dict[str, Any]]	Additional keyword arguments for text generation. These parameters override the parameters passed in the __init__ method. For more details, see OpenAI documentation.

Outputs

Parameter	Type	Description
replies	List[str]	A list of strings containing the generated responses.
meta	List[Dict[str, Any]]	A list of dictionaries containing metadata for each response, including model info and usage.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
api_key	Secret	Secret.from_env_var('OPENAI_API_KEY')	The OpenAI API key to connect to OpenAI.
model	str	gpt-5-mini	The name of the model to use.
streaming_callback	Optional[StreamingCallbackT]	None	A callback function called when a new token is received from the stream.
api_base_url	Optional[str]	None	An optional base URL.
organization	Optional[str]	None	The Organization ID. For help, see Setting up your organization.
system_prompt	Optional[str]	None	The system prompt to use for text generation. If not provided, the system prompt is omitted, and the default system prompt of the model is used.
generation_kwargs	Optional[Dict[str, Any]]	None	Other parameters to use for the model, sent directly to the OpenAI endpoint. See OpenAI documentation for more details.
timeout	Optional[float]	30.0	Timeout for OpenAI client calls. If not set, it is inferred from the OPENAI_TIMEOUT environment variable or set to 30.
max_retries	Optional[int]	five	Maximum retries to establish contact with OpenAI if it returns an internal error. If not set, it is inferred from the OPENAI_MAX_RETRIES environment variable or set to five.
http_client_kwargs	Optional[Dict[str, Any]]	None	A dictionary of keyword arguments to configure a custom httpx.Client or httpx.AsyncClient. For more information, see the HTTPX documentation.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
prompt	str		The string prompt to use for text generation.
system_prompt	Optional[str]	None	The system prompt to use for text generation. If this runtime system prompt is omitted, the system prompt defined at initialization time is used.
streaming_callback	Optional[StreamingCallbackT]	None	A callback function called when a new token is received from the stream.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for text generation. These parameters override the parameters passed in the __init__ method. For more details, see OpenAI documentation.

Related Information

• Use OpenAI Models