GoogleGenAIChatGenerator
Complete chats using Google's Gemini models through the Google Gen AI SDK.
Key Features
- Supports Gemini models (for example,
gemini-2.0-flash) through the unified Google Gen AI SDK. - Works with both the Gemini Developer API and Vertex AI through a single component.
- Returns responses in the
ChatMessageformat for chat-based pipelines. - Supports tool and function calling with
Toolobjects orToolset. - Configurable safety settings and generation parameters.
- Supports streaming for real-time token delivery.
Configuration
Authentication
You need a Google API key to use the Gemini Developer API. Connect Haystack Platform to your Google AI Studio account on the Integrations page. For details, see Use Google Gemini Models.
To use Vertex AI instead, set api to vertex and configure vertex_ai_project and vertex_ai_location. No API key is needed when using Application Default Credentials.
- Drag the
GoogleGenAIChatGeneratorcomponent onto the canvas from the Component Library. - Click the component to open the configuration panel.
- On the General tab:
- Enter the model name (for example,
gemini-2.0-flash). For available models, see the Google documentation.
- Enter the model name (for example,
- Go to the Advanced tab to configure the API type, API key, generation kwargs, Vertex AI project and location, safety settings, streaming callback, and tools.
Connections
GoogleGenAIChatGenerator accepts a list of ChatMessage instances as input (messages). It outputs a list of ChatMessage replies (replies).
Typically, you connect ChatPromptBuilder to the messages input to build the prompt. Connect the replies output to OutputAdapter and then to DeepsetAnswerBuilder to format the final answer.
Usage Example
Using the Component in a Pipeline
This is an example RAG pipeline with GoogleGenAIChatGenerator and DeepsetAnswerBuilder:
components:
bm25_retriever: # Selects the most similar documents from the document store
type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
init_parameters:
document_store:
type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
init_parameters:
hosts:
index: 'Standard-Index-English'
max_chunk_bytes: 104857600
embedding_dim: 768
return_embedding: false
method:
mappings:
settings:
create_index: true
http_auth:
use_ssl:
verify_certs:
timeout:
top_k: 20 # The number of results to return
fuzziness: 0
query_embedder:
type: deepset_cloud_custom_nodes.embedders.nvidia.text_embedder.DeepsetNvidiaTextEmbedder
init_parameters:
normalize_embeddings: true
model: intfloat/e5-base-v2
embedding_retriever: # Selects the most similar documents from the document store
type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
init_parameters:
document_store:
type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
init_parameters:
hosts:
index: 'Standard-Index-English'
max_chunk_bytes: 104857600
embedding_dim: 768
return_embedding: false
method:
mappings:
settings:
create_index: true
http_auth:
use_ssl:
verify_certs:
timeout:
top_k: 20 # The number of results to return
document_joiner:
type: haystack.components.joiners.document_joiner.DocumentJoiner
init_parameters:
join_mode: concatenate
ranker:
type: deepset_cloud_custom_nodes.rankers.nvidia.ranker.DeepsetNvidiaRanker
init_parameters:
model: intfloat/simlm-msmarco-reranker
top_k: 8
meta_field_grouping_ranker:
type: haystack.components.rankers.meta_field_grouping_ranker.MetaFieldGroupingRanker
init_parameters:
group_by: file_id
subgroup_by:
sort_docs_by: split_id
answer_builder:
type: deepset_cloud_custom_nodes.augmenters.deepset_answer_builder.DeepsetAnswerBuilder
init_parameters:
reference_pattern: acm
ChatPromptBuilder:
type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
init_parameters:
template:
- _content:
- text: "You are a helpful assistant answering the user's questions based on the provided documents.\nIf the answer is not in the documents, rely on the web_search tool to find information.\nDo not use your own knowledge.\n"
_role: system
- _content:
- text: "Provided documents:\n{% for document in documents %}\nDocument [{{ loop.index }}] :\n{{ document.content }}\n{% endfor %}\n\nQuestion: {{ query }}\n"
_role: user
required_variables:
variables:
OutputAdapter:
type: haystack.components.converters.output_adapter.OutputAdapter
init_parameters:
template: '{{ replies[0] }}'
output_type: List[str]
custom_filters:
unsafe: false
GoogleGenAIChatGenerator:
type: haystack_integrations.components.generators.google_genai.chat.chat_generator.GoogleGenAIChatGenerator
init_parameters:
api_key:
type: env_var
env_vars:
- GOOGLE_API_KEY
- GEMINI_API_KEY
strict: false
api: gemini
model: gemini-2.0-flash
generation_kwargs:
safety_settings:
streaming_callback:
tools:
connections: # Defines how the components are connected
- sender: bm25_retriever.documents
receiver: document_joiner.documents
- sender: query_embedder.embedding
receiver: embedding_retriever.query_embedding
- sender: embedding_retriever.documents
receiver: document_joiner.documents
- sender: document_joiner.documents
receiver: ranker.documents
- sender: ranker.documents
receiver: meta_field_grouping_ranker.documents
- sender: meta_field_grouping_ranker.documents
receiver: answer_builder.documents
- sender: meta_field_grouping_ranker.documents
receiver: ChatPromptBuilder.documents
- sender: OutputAdapter.output
receiver: answer_builder.replies
- sender: ChatPromptBuilder.prompt
receiver: GoogleGenAIChatGenerator.messages
- sender: GoogleGenAIChatGenerator.replies
receiver: OutputAdapter.replies
inputs: # Define the inputs for your pipeline
query: # These components will receive the query as input
- "bm25_retriever.query"
- "query_embedder.text"
- "ranker.query"
- "answer_builder.query"
- "ChatPromptBuilder.query"
filters: # These components will receive a potential query filter as input
- "bm25_retriever.filters"
- "embedding_retriever.filters"
outputs: # Defines the output of your pipeline
documents: "meta_field_grouping_ranker.documents" # The output of the pipeline is the retrieved documents
answers: "answer_builder.answers" # The output of the pipeline is the generated answers
max_runs_per_component: 100
metadata: {}
Parameters
Inputs
| Parameter | Type | Default | Description |
|---|---|---|---|
| messages | List[ChatMessage] | A list of ChatMessage instances representing the input messages. | |
| generation_kwargs | Optional[Dict[str, Any]] | None | Additional keyword arguments for the model. For details, see model documentation. |
| safety_settings | Optional[List[Dict[str, Any]]] | None | Safety settings for content filtering. If provided, it will override the default settings. |
| streaming_callback | Optional[StreamingCallbackT] | None | A callback function that is called when a new token is received from the stream. |
| tools | Optional[Union[List[Tool], Toolset]] | None | A list of Tool objects or a Toolset that the model can use. |
Outputs
| Parameter | Type | Default | Description |
|---|---|---|---|
| replies | List[ChatMessage] | A list containing the generated ChatMessage responses. |
Init Parameters
These are the parameters you can configure in Pipeline Builder:
| Parameter | Type | Default | Description |
|---|---|---|---|
| api_key | Secret | Secret.from_env_var(['GOOGLE_API_KEY', 'GEMINI_API_KEY'], strict=False) | Google API key, defaults to the GOOGLE_API_KEY and GEMINI_API_KEY environment variables. Not needed if using Vertex AI with Application Default Credentials. Go to Google AI Studio for a Gemini API key. Go to Google Cloud Vertex AI for a Vertex AI API key. |
| api | Literal['gemini', 'vertex'] | gemini | The API to use. Either "gemini" for the Gemini Developer API or "vertex" for Vertex AI. |
| vertex_ai_project | Optional[str] | None | Google Cloud project ID for Vertex AI. Required when using Vertex AI with Application Default Credentials. |
| vertex_ai_location | Optional[str] | None | Google Cloud location for Vertex AI (for example, "us-central1", "europe-west1"). Required when using Vertex AI with Application Default Credentials. |
| model | str | gemini-2.0-flash | Name of the model to use (for example, "gemini-2.0-flash") |
| generation_kwargs | Optional[Dict[str, Any]] | None | Configuration for generation (temperature, max_tokens, and more). |
| safety_settings | Optional[List[Dict[str, Any]]] | None | Safety settings for content filtering. |
| streaming_callback | Optional[StreamingCallbackT] | None | A callback function that is called when a new token is received from the stream. |
| tools | Optional[Union[List[Tool], Toolset]] | None | A list of Tool objects or a Toolset that the model can use. Each tool should have a unique name. |
Run Method Parameters
These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.
| Parameter | Type | Default | Description |
|---|---|---|---|
| messages | List[ChatMessage] | A list of ChatMessage instances representing the input messages. | |
| generation_kwargs | Optional[Dict[str, Any]] | None | Configuration for generation. |
| safety_settings | Optional[List[Dict[str, Any]]] | None | Safety settings for content filtering. |
| streaming_callback | Optional[StreamingCallbackT] | None | A callback function that is called when a new token is received from the stream. |
| tools | Optional[Union[List[Tool], Toolset]] | None | A list of Tool objects or a Toolset that the model can use. |
Was this page helpful?