OpenAIGenerator

Generates text using OpenAI's large language models (LLMs).

Basic Information

• Type: haystack_integrations.generators.openai.OpenAIGenerator

Inputs

Parameter	Type	Default	Description
prompt	str		The string prompt to use for text generation.
system_prompt	Optional[str]	None	The system prompt to use for text generation. If this run time system prompt is omitted, the system prompt, if defined at initialisation time, is used.
streaming_callback	Optional[StreamingCallbackT]	None	A callback function that is called when a new token is received from the stream.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for text generation. These parameters will potentially override the parameters passed in the __init__ method. For more details on the parameters supported by the OpenAI API, refer to the OpenAI documentation.

Outputs

Parameter	Type	Default	Description
replies	List[str]		A list of strings containing the generated responses and a list of dictionaries containing the metadata for each response.
meta	List[Dict[str, Any]]		A list of strings containing the generated responses and a list of dictionaries containing the metadata for each response.

Overview

Work in Progress

Bear with us while we're working on adding pipeline examples and most common components connections.

Generates text using OpenAI's large language models (LLMs).

It works with the gpt-4 and o-series models and supports streaming responses from OpenAI API. It uses strings as input and output.

You can customize how the text is generated by passing parameters to the OpenAI API. Use the **generation_kwargs argument when you initialize the component or when you run it. Any parameter that works with openai.ChatCompletion.create will work here too.

For details on OpenAI API parameters, see OpenAI documentation.

Usage Example

components:
  OpenAIGenerator:
    type: components.generators.openai.OpenAIGenerator
    init_parameters:

Parameters

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
api_key	Secret	Secret.from_env_var('OPENAI_API_KEY')	The OpenAI API key to connect to OpenAI.
model	str	gpt-4o-mini	The name of the model to use.
streaming_callback	Optional[StreamingCallbackT]	None	A callback function that is called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument.
api_base_url	Optional[str]	None	An optional base URL.
organization	Optional[str]	None	The Organization ID, defaults to None.
system_prompt	Optional[str]	None	The system prompt to use for text generation. If not provided, the system prompt is omitted, and the default system prompt of the model is used.
generation_kwargs	Optional[Dict[str, Any]]	None	Other parameters to use for the model. These parameters are all sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some of the supported parameters: - max_tokens: The maximum number of tokens the output text can have. - temperature: What sampling temperature to use. Higher values mean the model will take more risks. Try 0.9 for more creative applications and 0 (argmax sampling) for ones with a well-defined answer. - top_p: An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So, 0.1 means only the tokens comprising the top 10% probability mass are considered. - n: How many completions to generate for each prompt. For example, if the LLM gets 3 prompts and n is 2, it will generate two completions for each of the three prompts, ending up with 6 completions in total. - stop: One or more sequences after which the LLM should stop generating tokens. - presence_penalty: What penalty to apply if a token is already present at all. Bigger values mean the model will be less likely to repeat the same token in the text. - frequency_penalty: What penalty to apply if a token has already been generated in the text. Bigger values mean the model will be less likely to repeat the same token in the text. - logit_bias: Add a logit bias to specific tokens. The keys of the dictionary are tokens, and the values are the bias to add to that token.
timeout	Optional[float]	None	Timeout for OpenAI Client calls, if not set it is inferred from the OPENAI_TIMEOUT environment variable or set to 30.
max_retries	Optional[int]	None	Maximum retries to establish contact with OpenAI if it returns an internal error, if not set it is inferred from the OPENAI_MAX_RETRIES environment variable or set to 5.
http_client_kwargs	Optional[Dict[str, Any]]	None	A dictionary of keyword arguments to configure a custom httpx.Clientor httpx.AsyncClient. For more information, see the HTTPX documentation.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
prompt	str		The string prompt to use for text generation.
system_prompt	Optional[str]	None	The system prompt to use for text generation. If this run time system prompt is omitted, the system prompt, if defined at initialisation time, is used.
streaming_callback	Optional[StreamingCallbackT]	None	A callback function that is called when a new token is received from the stream.
generation_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for text generation. These parameters will potentially override the parameters passed in the __init__ method. For more details on the parameters supported by the OpenAI API, refer to the OpenAI documentation.