YAML Init Parameters

These are the parameters you can pass to this component in the pipeline YAML configuration:

Parameter	Type	Possible values	Description
`azure_endpoint`	String	Default: `None`	The endpoint of the deployed model, for example `https://example-resource.azure.openai.com/`. Optional.
`api_version`	String	Default: `2023-05-15`	The version of the API to use. Optional.
`azure_deployment`	String	Default: `gpt-35-turbo`	The deployment of the model, usually the model name. Optional.
`api_key`	Secret	Default: `{"type": "env_var", "env_vars": ["AZURE_OPENAI_API_KEY"], "strict": False}`	The API key to use for authentication. Optional.
`azure_ad_token`	Secret	Default: `{"type": "env_var", "env_vars": ["AZURE_OPENAI_AD_TOKEN"], "strict": False}`	Azure Active Directory token. Optional.
`organization`	String	Default: `None`	The Organization ID. For more information, see production best practices. Optional.
`streaming_callback`	Callable[StreamingChunk]	Default: `None`	A callback function that is called when a new token is received from the stream. The callback function accepts StreamingChunk as an argument. Optional.
`system_prompt`	String	Default: `None`	The prompt to use for the system. Optional.
`generation_kwargs`	Dictionary of string and any	Default: `{}`	Other parameters to use for the model. These parameters are all sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some of the supported parameters include: - `max_tokens`: The maximum number of tokens the output can have. - `temperature`: The sampling temperature to use. Higher values make the model more creative. Try 0.9 for more creative applications and 0 if you need a well-defined answer. - `top_p`: Nucleus sampling. It means the model considers the results of the tokens iwth top_p probablility mass. So 0.1 mean only the tokens comprising hte top 10% probabbility mass are considered. - `n`: The number of completions to generate for each prompt. For example, if an LLM gets three prompts and `n=2`, it will generate two completions for each prompt, resulting in six completions in total. - `stop`: One or more sequences after which the LLM should stop generating tokens. - `presence_penalty`: The penalty to apply if a token is already present. Higher values mean the model is less likely to repeat the same token in the text. - `frequency_penalty`: The penalty to apply if a token has already been generated. Higher values mean the model is less likely to repeat the same token. - `logit_bias`: Adds a logit bias to specific toens. The keys of the dictionary are tokens and the values are the bias to add to that token. Optional.
`max_retries`	Integer	Default: `5`	The maximum number of times AzureOpenAIGenerator tries to make a request. If not set, it's inferred from the OPENAI_MAX_RETRIES environment variable or set to `5`. Required.
`timeout`	Integer	Default: `30`	The timeout for the generator. If not set, it's inferred from the OPENAI_TIMEOUT environment variable or set to `30`. Required

REST API Runtime Parameters

There are no runtime parameters you can pass to this component when making a request to the Search REST API endpoint.