Learn how to customize OpenAIGenerator.
YAML Init Parameters
These are the parameters you can pass to this component in the pipeline YAML configuration:
Parameter | Type | Possible values | Description |
---|---|---|---|
api_key | Secret | Default: {"type": "env_var", "env_vars": ["OPENAI_API_KEY"], "strict": False} | The OpenAI API key. Required. |
model | String | Default: gpt-3.5-turbo | The name of the model to use. Required. |
streaming_callback | Callable[StreamingChunk] | Default: None | A callback function that is called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument. Optional. |
api_base_url | String | Default: None | An optional base URL. Optional. |
organization | String | Default: None | The Organization ID. For more information, see production best practices. Optional. |
system_prompt | String | Default: None | The system prompt to use for text generation. If not provided, the system prompt is omitted, and the default system prompt of the model is used. Optional. |
generation_kwargs | Dictionary of string and any | Default: None | Other parameters to use for the model. These parameters are all sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some supported parameters include: - max_tokens The maximum number of tokens the generated output can have.- temperature : The sampling temperature to use. Higher values mean the model takes more risks. Try 0.9 for more creative applications and 0 for well-defined answers.- top_p : An alternative to sampling with temperature. It's nucleus sampling where the model considers the results of the tokens with top_p probability mass. 0.1 means only the tokens comprising hte top 10% probability mass are considered.- n : The number of completions to generate for each prompt. For example, if the LLM gets three prompts and n=2 , it generates two completions for each prompt resulting in six completions in total.- stop : One or more sequences after which the LLM should stop generating tokens.- presence_penalty : The penalty to apply if a token is already present in the text of in the prompt. Higher values mean the model is less likely to repeat the same token in the generated text.- frequency_penalty : The penalty to apply if a token has alredy been generated in the text. Higher values mean the model is less likely to repeat the same token in the text.- logit_bias : Add a logit bias to specific tokens. The keys of the dictionary are tokens and the values are the bias to add to each token.Optional. |
timeout | Float | Default: None | Timeout for OpenAI Client calls. If not set, it is inferred from the OPENAI_TIMEOUT environment variable or set to 30.Optional. |
max_retries | Integer | Default: None | Maximum retries to establish contact with OpenAI if it returns an internal error. If not set, it is inferred from the OPENAI_MAX_RETRIES environment variable or set to 5.Optional. |
REST API Runtime Parameters
There are no runtime parameters you can pass to this component when making a request to the Search REST API endpoint.