OpenAIGenerator Parameters

Learn how to customize OpenAIGenerator.

YAML Init Parameters

These are the parameters you can pass to this component in the pipeline YAML configuration:

ParameterTypePossible valuesDescription
api_keySecretDefault: {"type": "env_var", "env_vars": ["OPENAI_API_KEY"], "strict": False}The OpenAI API key. Required.
modelStringDefault: gpt-3.5-turboThe name of the model to use. Required.
streaming_callbackCallable[StreamingChunk]Default: NoneA callback function that is called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument.
Optional.
api_base_urlStringDefault: NoneAn optional base URL. Optional.
organizationStringDefault: NoneThe Organization ID. For more information, see production best practices.
Optional.
system_promptStringDefault: NoneThe system prompt to use for text generation. If not provided, the system prompt is omitted, and the default system prompt of the model is used.
Optional.
generation_kwargsDictionary of string and anyDefault: NoneOther parameters to use for the model. These parameters are all sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some supported parameters include:

- max_tokens The maximum number of tokens the generated output can have.
- temperature: The sampling temperature to use. Higher values mean the model takes more risks. Try 0.9 for more creative applications and 0 for well-defined answers.
- top_p: An alternative to sampling with temperature. It's nucleus sampling where the model considers the results of the tokens with top_p probability mass. 0.1 means only the tokens comprising hte top 10% probability mass are considered.
- n: The number of completions to generate for each prompt. For example, if the LLM gets three prompts and n=2, it generates two completions for each prompt resulting in six completions in total.
- stop: One or more sequences after which the LLM should stop generating tokens.
- presence_penalty: The penalty to apply if a token is already present in the text of in the prompt. Higher values mean the model is less likely to repeat the same token in the generated text.
- frequency_penalty: The penalty to apply if a token has alredy been generated in the text. Higher values mean the model is less likely to repeat the same token in the text.
- logit_bias: Add a logit bias to specific tokens. The keys of the dictionary are tokens and the values are the bias to add to each token.
Optional.
timeoutFloatDefault: NoneTimeout for OpenAI Client calls. If not set, it is inferred from the OPENAI_TIMEOUT environment variable or set to 30.
Optional.
max_retriesIntegerDefault: NoneMaximum retries to establish contact with OpenAI if it returns an internal error. If not set, it is inferred from the OPENAI_MAX_RETRIES environment variable or set to 5.
Optional.


REST API Runtime Parameters

There are no runtime parameters you can pass to this component when making a request to the Search REST API endpoint.