VertexAIImageCaptioner

VertexAIImageCaptioner enables text generation using Google Vertex AI imagetext generative model.

Basic Information

• Type: haystack_integrations.components.generators.google_vertex.captioner.VertexAIImageCaptioner

Inputs

Parameter	Type	Default	Description
image	ByteStream		The image to generate captions for.

Outputs

Parameter	Type	Default	Description
captions	List[str]		A dictionary with the following keys: - captions: A list of captions generated by the model.

Overview

Work in Progress

Bear with us while we're working on adding pipeline examples and most common components connections.

VertexAIImageCaptioner enables text generation using Google Vertex AI imagetext generative model.

Authenticates using Google Cloud Application Default Credentials (ADCs). For more information see the official Google documentation.

Usage example:

import requests

from haystack.dataclasses.byte_stream import ByteStream
from haystack_integrations.components.generators.google_vertex import VertexAIImageCaptioner

captioner = VertexAIImageCaptioner()

image = ByteStream(
    data=requests.get(
        "https://raw.githubusercontent.com/deepset-ai/haystack-core-integrations/main/integrations/google_vertex/example_assets/robot1.jpg"
    ).content
)
result = captioner.run(image=image)

for caption in result["captions"]:
    print(caption)

>>> two gold robots are standing next to each other in the desert

Usage Example

components:
  VertexAIImageCaptioner:
    type: google_vertex.src.haystack_integrations.components.generators.google_vertex.captioner.VertexAIImageCaptioner
    init_parameters:

Parameters

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
project_id	Optional[str]	None	ID of the GCP project to use. By default, it is set during Google Cloud authentication.
model	str	imagetext	Name of the model to use.
location	Optional[str]	None	The default location to use when making API calls, if not set uses us-central-1. Defaults to None.
kwargs	Any		Additional keyword arguments to pass to the model. For a list of supported arguments see the ImageTextModel.get_captions() documentation.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
image	ByteStream		The image to generate captions for.