AzureOpenAIGenerator Parameters

Check the init and runtime parameters you can use to customize AzureOpenAIGenerator.

YAML Init Parameters

These are the parameters you can pass to this component in the pipeline YAML configuration:

ParameterTypePossible valuesDescription
azure_endpointStringDefault: NoneThe endpoint of the deployed model, for example https://example-resource.azure.openai.com/. Optional.
api_versionStringDefault: 2023-05-15The version of the API to use. Optional.
azure_deploymentStringDefault: gpt-35-turboThe deployment of the model, usually the model name.
Optional.
api_keySecretDefault: {"type": "env_var", "env_vars": ["AZURE_OPENAI_API_KEY"], "strict": False}The API key to use for authentication.
Optional.
azure_ad_tokenSecretDefault: {"type": "env_var", "env_vars": ["AZURE_OPENAI_AD_TOKEN"], "strict": False}Azure Active Directory token.
Optional.
organizationStringDefault: NoneThe Organization ID. For more information, see production best practices.
Optional.
streaming_callbackCallable[StreamingChunk]Default: NoneA callback function that is called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument.
Optional.
system_promptStringDefault: NoneThe prompt to use for the system.
Optional.
generation_kwargsDictionary of string and anyDefault: {}Other parameters to use for the model. These parameters are all sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some of the supported parameters include:

- max_tokens: The maximum number of tokens the output can have.
- temperature: The sampling temperature to use. Higher values make the model more creative. Try 0.9 for more creative applications and 0 if you need a well-defined answer.
- top_p: Nucleus sampling. It means the model considers the results of the tokens iwth top_p probablility mass. So 0.1 mean only the tokens comprising hte top 10% probabbility mass are considered.
- n: The number of completions to generate for each prompt. For example, if an LLM gets three prompts and n=2, it will generate two completions for each prompt, resulting in six completions in total.
- stop: One or more sequences after which the LLM should stop generating tokens.
- presence_penalty: The penalty to apply if a token is already present. Higher values mean the model is less likely to repeat the same token in the text.
- frequency_penalty: The penalty to apply if a token has already been generated. Higher values mean the model is less likely to repeat the same token.
- logit_bias: Adds a logit bias to specific toens. The keys of the dictionary are tokens and the values are the bias to add to that token.
Optional.
max_retriesIntegerDefault: 5The maximum number of times AzureOpenAIGenerator tries to make a request. If not set, it's inferred from the OPENAI_MAX_RETRIES environment variable or set to 5.
Required.
timeoutIntegerDefault: 30The timeout for the generator. If not set, it's inferred from the OPENAI_TIMEOUT environment variable or set to 30.
Required

REST API Runtime Parameters

There are no runtime parameters you can pass to this component when making a request to the Search REST API endpoint.