ImageFileToImageContent

Convert image files to ImageContent objects for multimodal AI processing, including tasks like image captioning or visual question answering.

Key Features

• Converts image files to ImageContent objects containing base64-encoded image data and metadata
• Supports various image formats
• Optionally resizes images while maintaining aspect ratio to reduce file size, memory usage, and processing time
• Configurable detail level for optimization with different AI models (such as OpenAI)
• Detail level and size can be set at init time and overridden at run time
• Accepts file paths and ByteStream objects as input

Configuration

1. Drag the ImageFileToImageContent component onto the canvas from the Component Library.
2. Click the component to open the configuration panel.
3. Configure the parameters as needed.

Connections

ImageFileToImageContent receives image file paths or ByteStream objects as input — typically from FilesInput. It outputs a list of ImageContent objects that you can send to ChatPromptBuilder to include images in multimodal prompts for a vision-enabled model.

Usage Example

Using the Component in a Pipeline

This is an example query pipeline that uses ImageFileToImageContent to convert uploaded images to ImageContent objects for multimodal processing. The images are sent to a ChatPromptBuilder along with a user question, and then processed by a vision-enabled chat generator:

components:
  ImageFileToImageContent:
    type: haystack.components.converters.image.ImageFileToImageContent
    init_parameters:
      detail: auto
      size:

  ChatPromptBuilder:
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
    init_parameters:
      template:
      - _content:
        - text: "You are an AI assistant that can analyze images and answer questions about them."
        _role: system
      - _content:
        - text: "{% for image in images %}{{ image }}{% endfor %}\n\nQuestion: {{ query }}"
        _role: user
      required_variables:
      variables:

  OutputAdapter:
    type: haystack.components.converters.output_adapter.OutputAdapter
    init_parameters:
      template: '{{ replies[0] }}'
      output_type: List[str]
      custom_filters:
      unsafe: false

  answer_builder:
    type: deepset_cloud_custom_nodes.augmenters.deepset_answer_builder.DeepsetAnswerBuilder
    init_parameters:
      reference_pattern: acm

  OpenAIChatGenerator:
    type: haystack.components.generators.chat.openai.OpenAIChatGenerator
    init_parameters:
      api_key:
        type: env_var
        env_vars:
        - OPENAI_API_KEY
        strict: false
      model: gpt-4o
      generation_kwargs:
      streaming_callback:
      tools:

connections:
- sender: ImageFileToImageContent.image_contents
  receiver: ChatPromptBuilder.images
- sender: ChatPromptBuilder.prompt
  receiver: OpenAIChatGenerator.messages
- sender: OpenAIChatGenerator.replies
  receiver: OutputAdapter.replies
- sender: OutputAdapter.output
  receiver: answer_builder.replies

inputs:
  query:
  - ChatPromptBuilder.query
  - answer_builder.query
  files:
  - ImageFileToImageContent.sources

outputs:
  answers: answer_builder.answers

max_runs_per_component: 100

metadata: {}

Parameters

Inputs

Parameter	Type	Default	Description
sources	List[Union[str, Path, ByteStream]]		List of image file paths or ByteStream objects to convert.
meta	Optional[Union[Dict[str, Any], List[Dict[str, Any]]]]	None	Optional metadata to attach to the ImageContent objects. This value can be a list of dictionaries or a single dictionary. If it's a single dictionary, its content is added to the metadata of all produced ImageContent objects. If it's a list, its length must match the number of sources as they're zipped together. For ByteStream objects, their meta is added to the output ImageContent objects.
detail	Optional[Literal["auto", "high", "low"]]	None	Optional detail level of the image (only supported by OpenAI). This is passed to the created ImageContent objects. If not provided, the detail level is the one set in the constructor.
size	Optional[Tuple[int, int]]	None	If provided, resizes the image to fit within the specified dimensions (width, height) while maintaining aspect ratio. If not provided, the size value is the one set in the constructor.

Outputs

Parameter	Type	Default	Description
image_contents	List[ImageContent]		A list of ImageContent objects created from the input image files.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
detail	Optional[Literal["auto", "high", "low"]]	None	Optional detail level of the image (only supported by OpenAI). Possible values: - "auto" - "high" - "low" This is passed to the created ImageContent objects.
size	Optional[Tuple[int, int]]	None	If provided, resizes the image to fit within the specified dimensions (width, height) while maintaining aspect ratio. This reduces file size, memory usage, and processing time.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
sources	List[Union[str, Path, ByteStream]]		List of image file paths or ByteStream objects to convert.
meta	Optional[Union[Dict[str, Any], List[Dict[str, Any]]]]	None	Optional metadata to attach to the ImageContent objects. This value can be a list of dictionaries or a single dictionary. If it's a single dictionary, its content is added to the metadata of all produced ImageContent objects. If it's a list, its length must match the number of sources as they're zipped together. For ByteStream objects, their meta is added to the output ImageContent objects.
detail	Optional[Literal["auto", "high", "low"]]	None	Optional detail level of the image (only supported by OpenAI). This will be passed to the created ImageContent objects. If not provided, the detail level will be the one set in the constructor.
size	Optional[Tuple[int, int]]	None	If provided, resizes the image to fit within the specified dimensions (width, height) while maintaining aspect ratio. If not provided, the size value will be the one set in the constructor.