AzureOpenAIGenerator

Generate text using OpenAI's large language models (LLMs).

Basic Information

• Type: haystack_integrations.generators.azure.AzureOpenAIGenerator

Inputs

Parameter	Type	Default	Description

Outputs

Parameter	Type	Default	Description

Overview

Work in Progress

Bear with us while we're working on adding pipeline examples and most common components connections.

Generates text using OpenAI's large language models (LLMs).

It works with the gpt-4 - type models and supports streaming responses from OpenAI API.

You can customize how the text is generated by passing parameters to the OpenAI API. Use the **generation_kwargs argument when you initialize the component or when you run it. Any parameter that works with openai.ChatCompletion.create will work here too.

For details on OpenAI API parameters, see OpenAI documentation.

Usage Example

components:
  AzureOpenAIGenerator:
    type: components.generators.azure.AzureOpenAIGenerator
    init_parameters:

Parameters

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
azure_endpoint	Optional[str]	None	The endpoint of the deployed model, for example https://example-resource.azure.openai.com/.
api_version	Optional[str]	2023-05-15	The version of the API to use. Defaults to 2023-05-15.
azure_deployment	Optional[str]	gpt-4o-mini	The deployment of the model, usually the model name.
api_key	Optional[Secret]	Secret.from_env_var('AZURE_OPENAI_API_KEY', strict=False)	The API key to use for authentication.
azure_ad_token	Optional[Secret]	Secret.from_env_var('AZURE_OPENAI_AD_TOKEN', strict=False)	Azure Active Directory token.
organization	Optional[str]	None	Your organization ID, defaults to None. For help, see Setting up your organization.
streaming_callback	Optional[StreamingCallbackT]	None	A callback function called when a new token is received from the stream. It accepts StreamingChunk as an argument.
system_prompt	Optional[str]	None	The system prompt to use for text generation. If not provided, the Generator omits the system prompt and uses the default system prompt.
timeout	Optional[float]	None	Timeout for AzureOpenAI client. If not set, it is inferred from the OPENAI_TIMEOUT environment variable or set to 30.
max_retries	Optional[int]	None	Maximum retries to establish contact with AzureOpenAI if it returns an internal error. If not set, it is inferred from the OPENAI_MAX_RETRIES environment variable or set to 5.
http_client_kwargs	Optional[Dict[str, Any]]	None	A dictionary of keyword arguments to configure a custom httpx.Clientor httpx.AsyncClient. For more information, see the HTTPX documentation.
generation_kwargs	Optional[Dict[str, Any]]	None	Other parameters to use for the model, sent directly to the OpenAI endpoint. See OpenAI documentation for more details. Some of the supported parameters: - max_tokens: The maximum number of tokens the output text can have. - temperature: The sampling temperature to use. Higher values mean the model takes more risks. Try 0.9 for more creative applications and 0 (argmax sampling) for ones with a well-defined answer. - top_p: An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. For example, 0.1 means only the tokens comprising the top 10% probability mass are considered. - n: The number of completions to generate for each prompt. For example, with 3 prompts and n=2, the LLM will generate two completions per prompt, resulting in 6 completions total. - stop: One or more sequences after which the LLM should stop generating tokens. - presence_penalty: The penalty applied if a token is already present. Higher values make the model less likely to repeat the token. - frequency_penalty: Penalty applied if a token has already been generated. Higher values make the model less likely to repeat the token. - logit_bias: Adds a logit bias to specific tokens. The keys of the dictionary are tokens, and the values are the bias to add to that token.
default_headers	Optional[Dict[str, str]]	None	Default headers to use for the AzureOpenAI client.
azure_ad_token_provider	Optional[AzureADTokenProvider]	None	A function that returns an Azure Active Directory token, will be invoked on every request.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description