VertexAIImageQA

Answer questions about images using Google Vertex AI generative models.

Basic Information

• Type: haystack_integrations.components.generators.google_vertex.question_answering.VertexAIImageQA
• Components it can connect with:
  • Input: Receives an image as ByteStream and a question as input.
  • AnswerBuilder: Sends generated answers to AnswerBuilder.

Inputs

Parameter	Type	Default	Description
image	ByteStream		The image to ask the question about.
question	str		The question to ask.

Outputs

Parameter	Type	Default	Description
replies	List[str]		A list of answers to the question.

Overview

VertexAIImageQA answers questions about images using Google Vertex AI generative models. It takes an image and a question, then returns answers based on the image content.

Authorization

This component authenticates using Google Cloud Application Default Credentials (ADCs). For more information, see the official Google documentation.

Create secrets for GCP_PROJECT_ID and optionally GCP_DEFAULT_REGION. For detailed instructions on creating secrets, see Create Secrets.

Usage Example

This pipeline uses VertexAIImageQA to answer questions about images:

components:
  VertexAIImageQA:
    type: haystack_integrations.components.generators.google_vertex.question_answering.VertexAIImageQA
    init_parameters:
      project_id:
      model: imagetext
      location:

  AnswerBuilder:
    type: haystack.components.builders.answer_builder.AnswerBuilder
    init_parameters:
      pattern:
      reference_pattern:

connections:
- sender: VertexAIImageQA.replies
  receiver: AnswerBuilder.replies

outputs:
  answers: AnswerBuilder.answers

max_runs_per_component: 100

metadata: {}

inputs:
  query:
  - VertexAIImageQA.question
  - AnswerBuilder.query

Parameters

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
project_id	Optional[str]	None	ID of the GCP project to use. By default, it is set during Google Cloud authentication.
model	str	imagetext	Name of the model to use.
location	Optional[str]	None	The default location to use when making API calls. If not set, uses us-central-1.
kwargs	Any		Additional keyword arguments to pass to the model. See the ImageTextModel.ask_question() documentation.

Run Method Parameters

These are the parameters you can configure for the run() method. You can pass these parameters at query time through the API, in Playground, or when running a job.

Parameter	Type	Default	Description
image	ByteStream		The image to ask the question about.
question	str		The question to ask.