Check the init and runtime parameters you can use to customize AzureOpenAIGenerator.
YAML Init Parameters
These are the parameters you can pass to this component in the pipeline YAML configuration:
Parameter | Type | Possible values | Description |
---|---|---|---|
azure_endpoint | String | Default: None | The endpoint of the deployed model, for example https://example-resource.azure.openai.com/ . Optional. |
api_version | String | Default: 2023-05-15 | The version of the API to use. Optional. |
azure_deployment | String | Default: gpt-35-turbo | The deployment of the model, usually the model name. Optional. |
api_key | Secret | Default: {"type": "env_var", "env_vars": ["AZURE_OPENAI_API_KEY"], "strict": False} | The API key to use for authentication. Optional. |
azure_ad_token | Secret | Default: {"type": "env_var", "env_vars": ["AZURE_OPENAI_AD_TOKEN"], "strict": False} | Azure Active Directory token. Optional. |
organization | String | Default: None | The Organization ID. For more information, see production best practices. Optional. |
streaming_callback | Callable[StreamingChunk] | Default: None | A callback function that is called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument. Optional. |
system_prompt | String | Default: None | The prompt to use for the system. Optional. |
generation_kwargs | Dictionary of string and any | Default: {} | Other parameters to use for the model. These parameters are all sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some of the supported parameters include: - max_tokens : The maximum number of tokens the output can have.- temperature : The sampling temperature to use. Higher values make the model more creative. Try 0.9 for more creative applications and 0 if you need a well-defined answer.- top_p : Nucleus sampling. It means the model considers the results of the tokens iwth top_p probablility mass. So 0.1 mean only the tokens comprising hte top 10% probabbility mass are considered.- n : The number of completions to generate for each prompt. For example, if an LLM gets three prompts and n=2 , it will generate two completions for each prompt, resulting in six completions in total.- stop : One or more sequences after which the LLM should stop generating tokens.- presence_penalty : The penalty to apply if a token is already present. Higher values mean the model is less likely to repeat the same token in the text.- frequency_penalty : The penalty to apply if a token has already been generated. Higher values mean the model is less likely to repeat the same token.- logit_bias : Adds a logit bias to specific toens. The keys of the dictionary are tokens and the values are the bias to add to that token.Optional. |
max_retries | Integer | Default: 5 | The maximum number of times AzureOpenAIGenerator tries to make a request. If not set, it's inferred from the OPENAI_MAX_RETRIES environment variable or set to 5 .Required. |
timeout | Integer | Default: 30 | The timeout for the generator. If not set, it's inferred from the OPENAI_TIMEOUT environment variable or set to 30 .Required |
REST API Runtime Parameters
There are no runtime parameters you can pass to this component when making a request to the Search REST API endpoint.