NvidiaGenerator

Generates text using generative models hosted with

Basic Information

• Type: haystack_integrations.nvidia.src.haystack_integrations.components.generators.nvidia.generator.NvidiaGenerator

Inputs

Parameter	Type	Default	Description
prompt	str		Text to be sent to the generative model.

Outputs

Parameter	Type	Default	Description
replies	List[str]		A dictionary with the following keys: - replies - Replies generated by the model. - meta - Metadata for each reply.
meta	List[Dict[str, Any]]		A dictionary with the following keys: - replies - Replies generated by the model. - meta - Metadata for each reply.

Overview

Work in Progress

Bear with us while we're working on adding pipeline examples and most common components connections.

Generates text using generative models hosted with NVIDIA NIM on the NVIDIA API Catalog.You need an NVIDIA API key for this component to work.

Usage Example

components:
  NvidiaGenerator:
    type: nvidia.src.haystack_integrations.components.generators.nvidia.generator.NvidiaGenerator
    init_parameters:

Parameters

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
model	Optional[str]	None	Name of the model to use for text generation. See the NVIDIA NIMs for more information on the supported models. Note: If no specific model along with locally hosted API URL is provided, the system defaults to the available model found using /models API. Check supported models at NVIDIA NIM.
api_key	Optional[Secret]	Secret.from_env_var('NVIDIA_API_KEY')	API key for the NVIDIA NIM. Set it as the NVIDIA_API_KEY environment variable or pass it here.
api_url	str	os.getenv('NVIDIA_API_URL', DEFAULT_API_URL)	Custom API URL for the NVIDIA NIM.
model_arguments	Optional[Dict[str, Any]]	None	Additional arguments to pass to the model provider. These arguments are specific to a model. Search your model in the NVIDIA NIM to find the arguments it accepts.
timeout	Optional[float]	None	Timeout for request calls, if not set it is inferred from the NVIDIA_TIMEOUT environment variable or set to 60 by default.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
prompt	str		Text to be sent to the generative model.