GoogleGenAIChatGenerator
Complete chats using Google's Gemini models through the Google Gen AI SDK.
Key Features
- Unified interface for both the Gemini Developer API and Vertex AI
- Supports Gemini 2.0 Flash and other Gemini model variants
- Streaming support for real-time token-by-token responses
- Tool/function calling support
- Configurable generation parameters such as temperature and max_tokens
- Configurable safety settings for content filtering
Configuration
- Drag the
GoogleGenAIChatGeneratorcomponent onto the canvas from the Component Library. - Click on the component to open the configuration panel.
- On the General tab:
- Choose the API to use:
geminifor the Gemini Developer API orvertexfor Vertex AI. - For the Gemini Developer API, enter your Google API key. Get your API key from Google AI Studio. For Vertex AI, enter your GCP project ID and location instead.
- Select a model (for example,
gemini-2.0-flash). For details, see Use Google Gemini Models.
- Choose the API to use:
- Go to the Advanced tab to configure generation parameters, safety settings, tools, and streaming.
Connections
GoogleGenAIChatGenerator accepts a list of ChatMessage objects through its messages input and outputs generated responses as replies (a list of ChatMessage instances).
Connect ChatPromptBuilder's prompt output to this component's messages input. Connect the replies output to DeepsetAnswerBuilder through OutputAdapter.
Source Code
To check this component's source code, open chat_generator.py in the Haystack Core Integrations repository.
Usage Examples
Basic Configuration
GoogleGenAIChatGenerator:
type: haystack_integrations.components.generators.google_genai.chat.chat_generator.GoogleGenAIChatGenerator
init_parameters:
api_key:
type: env_var
env_vars:
- GOOGLE_API_KEY
- GEMINI_API_KEY
strict: false
api: gemini
model: gemini-2.0-flash
Using the Component in a Pipeline
This is an example RAG pipeline with GoogleGenAIChatGenerator and DeepsetAnswerBuilder:
components:
bm25_retriever: # Selects the most similar documents from the document store
type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
init_parameters:
document_store:
type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
init_parameters:
hosts:
index: 'Standard-Index-English'
max_chunk_bytes: 104857600
embedding_dim: 768
return_embedding: false
method:
mappings:
settings:
create_index: true
http_auth:
use_ssl:
verify_certs:
timeout:
top_k: 20 # The number of results to return
fuzziness: 0
query_embedder:
type: deepset_cloud_custom_nodes.embedders.nvidia.text_embedder.DeepsetNvidiaTextEmbedder
init_parameters:
normalize_embeddings: true
model: intfloat/e5-base-v2
embedding_retriever: # Selects the most similar documents from the document store
type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
init_parameters:
document_store:
type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
init_parameters:
hosts:
index: 'Standard-Index-English'
max_chunk_bytes: 104857600
embedding_dim: 768
return_embedding: false
method:
mappings:
settings:
create_index: true
http_auth:
use_ssl:
verify_certs:
timeout:
top_k: 20 # The number of results to return
document_joiner:
type: haystack.components.joiners.document_joiner.DocumentJoiner
init_parameters:
join_mode: concatenate
ranker:
type: deepset_cloud_custom_nodes.rankers.nvidia.ranker.DeepsetNvidiaRanker
init_parameters:
model: intfloat/simlm-msmarco-reranker
top_k: 8
meta_field_grouping_ranker:
type: haystack.components.rankers.meta_field_grouping_ranker.MetaFieldGroupingRanker
init_parameters:
group_by: file_id
subgroup_by:
sort_docs_by: split_id
answer_builder:
type: deepset_cloud_custom_nodes.augmenters.deepset_answer_builder.DeepsetAnswerBuilder
init_parameters:
reference_pattern: acm
ChatPromptBuilder:
type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
init_parameters:
template:
- _content:
- text: "You are a helpful assistant answering the user's questions based on the provided documents.\nIf the answer is not in the documents, rely on the web_search tool to find information.\nDo not use your own knowledge.\n"
_role: system
- _content:
- text: "Provided documents:\n{% for document in documents %}\nDocument [{{ loop.index }}] :\n{{ document.content }}\n{% endfor %}\n\nQuestion: {{ query }}\n"
_role: user
required_variables:
variables:
OutputAdapter:
type: haystack.components.converters.output_adapter.OutputAdapter
init_parameters:
template: '{{ replies[0] }}'
output_type: List[str]
custom_filters:
unsafe: false
GoogleGenAIChatGenerator:
type: haystack_integrations.components.generators.google_genai.chat.chat_generator.GoogleGenAIChatGenerator
init_parameters:
api_key:
type: env_var
env_vars:
- GOOGLE_API_KEY
- GEMINI_API_KEY
strict: false
api: gemini
model: gemini-2.0-flash
generation_kwargs:
safety_settings:
streaming_callback:
tools:
connections: # Defines how the components are connected
- sender: bm25_retriever.documents
receiver: document_joiner.documents
- sender: query_embedder.embedding
receiver: embedding_retriever.query_embedding
- sender: embedding_retriever.documents
receiver: document_joiner.documents
- sender: document_joiner.documents
receiver: ranker.documents
- sender: ranker.documents
receiver: meta_field_grouping_ranker.documents
- sender: meta_field_grouping_ranker.documents
receiver: answer_builder.documents
- sender: meta_field_grouping_ranker.documents
receiver: ChatPromptBuilder.documents
- sender: OutputAdapter.output
receiver: answer_builder.replies
- sender: ChatPromptBuilder.prompt
receiver: GoogleGenAIChatGenerator.messages
- sender: GoogleGenAIChatGenerator.replies
receiver: OutputAdapter.replies
inputs: # Define the inputs for your pipeline
query: # These components will receive the query as input
- "bm25_retriever.query"
- "query_embedder.text"
- "ranker.query"
- "answer_builder.query"
- "ChatPromptBuilder.query"
filters: # These components will receive a potential query filter as input
- "bm25_retriever.filters"
- "embedding_retriever.filters"
outputs: # Defines the output of your pipeline
documents: "meta_field_grouping_ranker.documents" # The output of the pipeline is the retrieved documents
answers: "answer_builder.answers" # The output of the pipeline is the generated answers
max_runs_per_component: 100
metadata: {}
Parameters
Inputs
| Parameter | Type | Default | Description |
|---|---|---|---|
messages | List[ChatMessage] | A list of ChatMessage instances representing the input messages. | |
generation_kwargs | Optional[Dict[str, Any]] | None | Additional keyword arguments for the model. |
safety_settings | Optional[List[Dict[str, Any]]] | None | Safety settings for content filtering. If provided, overrides the default settings. |
streaming_callback | Optional[StreamingCallbackT] | None | A callback function that is called when a new token is received from the stream. |
tools | Optional[Union[List[Tool], Toolset]] | None | A list of Tool objects or a Toolset that the model can use. |
Outputs
| Parameter | Type | Description |
|---|---|---|
replies | List[ChatMessage] | A list containing the generated ChatMessage responses. |
Init Parameters
These are the parameters you can configure in Pipeline Builder:
| Parameter | Type | Default | Description |
|---|---|---|---|
api_key | Secret | Secret.from_env_var(['GOOGLE_API_KEY', 'GEMINI_API_KEY'], strict=False) | Google API key, defaults to the GOOGLE_API_KEY and GEMINI_API_KEY environment variables. Not needed if using Vertex AI with Application Default Credentials. Go to Google AI Studio for a Gemini API key. Go to Google Cloud Vertex AI for a Vertex AI API key. |
api | Literal['gemini', 'vertex'] | gemini | The API to use. Either "gemini" for the Gemini Developer API or "vertex" for Vertex AI. |
vertex_ai_project | Optional[str] | None | Google Cloud project ID for Vertex AI. Required when using Vertex AI with Application Default Credentials. |
vertex_ai_location | Optional[str] | None | Google Cloud location for Vertex AI (for example, "us-central1", "europe-west1"). Required when using Vertex AI with Application Default Credentials. |
model | str | gemini-2.0-flash | Name of the model to use (for example, "gemini-2.0-flash"). |
generation_kwargs | Optional[Dict[str, Any]] | None | Configuration for generation (temperature, max_tokens, and more). |
safety_settings | Optional[List[Dict[str, Any]]] | None | Safety settings for content filtering. |
streaming_callback | Optional[StreamingCallbackT] | None | A callback function that is called when a new token is received from the stream. |
tools | Optional[Union[List[Tool], Toolset]] | None | A list of Tool objects or a Toolset that the model can use. Each tool should have a unique name. |
Run Method Parameters
These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.
| Parameter | Type | Default | Description |
|---|---|---|---|
messages | List[ChatMessage] | A list of ChatMessage instances representing the input messages. | |
generation_kwargs | Optional[Dict[str, Any]] | None | Configuration for generation. |
safety_settings | Optional[List[Dict[str, Any]]] | None | Safety settings for content filtering. |
streaming_callback | Optional[StreamingCallbackT] | None | A callback function that is called when a new token is received from the stream. |
tools | Optional[Union[List[Tool], Toolset]] | None | A list of Tool objects or a Toolset that the model can use. |
Related Information
Was this page helpful?