TwelveLabsVideoConverter

Convert videos to Haystack Document objects using TwelveLabs Pegasus. Pegasus analyzes video visuals and audio in one step and returns a text description or transcript, so you do not need separate frame extraction or transcription components.

Key Features

• Converts each source video into one document whose content is Pegasus's analysis.
• Accepts publicly accessible video URLs or local file paths (up to 200 MB for direct upload).
• Supports configurable analysis prompts, temperature, and output length.
• Adds metadata such as source, asset_id, analysis_id, and model to each document.

Configuration

Add Workspace-Level Integration

1. Click your profile icon and choose Settings.
2. Go to Workspace>Integrations.
3. Find the provider you want to connect and click Connect next to them.
4. Enter the API key and any other required details.
5. Click Connect. You can use this integration in pipelines and indexes in the current workspace.

Add Organization-Level Integration

1. Click your profile icon and choose Settings.
2. Go to Organization>Integrations.
3. Find the provider you want to connect and click Connect next to them.
4. Enter the API key and any other required details.
5. Click Connect. You can use this integration in pipelines and indexes in all workspaces in the current organization.

1. Drag the TwelveLabsVideoConverter component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   1. Connect Haystack Platform to TwelveLabs by creating a secret called TWELVELABS_API_KEY. For more information about secrets, see Create Secrets.
   2. Set the model to pegasus1.5 or pegasus1.2.
   3. Optionally customize the prompt sent to Pegasus for each video.
4. Go to the Advanced tab to configure temperature and max_tokens.

Connections

TwelveLabsVideoConverter receives a list of video sources as strings (URLs or local file paths). It outputs a list of Document objects you can connect to a splitter, embedder, or document writer.

Source Code

To check this component's source code, open video_converter.py in the Haystack Core Integrations repository.

Usage Examples

Basic Configuration

  TwelveLabsVideoConverter:
    type: haystack_integrations.components.converters.twelvelabs.video_converter.TwelveLabsVideoConverter
    init_parameters:
      api_key:
        type: env_var
        env_vars:
        - TWELVELABS_API_KEY
        strict: false
      model: pegasus1.5

Using the Component in a Pipeline

# haystack-pipeline
components:
  TwelveLabsVideoConverter:
    type: haystack_integrations.components.converters.twelvelabs.video_converter.TwelveLabsVideoConverter
    init_parameters:
      api_key:
        type: env_var
        env_vars:
        - TWELVELABS_API_KEY
        strict: false
      model: pegasus1.5

  document_splitter:
    type: haystack.components.preprocessors.document_splitter.DocumentSplitter
    init_parameters:
      split_by: word
      split_length: 200
      split_overlap: 20

connections:
- sender: TwelveLabsVideoConverter.documents
  receiver: document_splitter.documents

max_runs_per_component: 100

metadata: {}

Parameters

Inputs

Parameter	Type	Description
sources	List[str]	Video sources as publicly accessible direct video URLs or local file paths.
meta	Optional[Union[Dict[str, Any], List[Dict[str, Any]]]]	Optional metadata to attach to the produced documents. Either a single dict applied to all sources, or a list aligned with sources.

Outputs

Parameter	Type	Description
documents	List[Document]	One document per successfully analyzed video.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
api_key	Secret	Secret.from_env_var("TWELVELABS_API_KEY")	The TwelveLabs API key.
model	str	pegasus1.5	The Pegasus model name (pegasus1.5 or pegasus1.2).
prompt	str	Default analysis prompt	The analysis prompt sent to Pegasus for each video.
temperature	float	0.2	Sampling temperature (0 to 1).
max_tokens	int	16384	Maximum output tokens per analysis.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
sources	List[str]		Video sources as URLs or local file paths.
meta	Optional[Union[Dict[str, Any], List[Dict[str, Any]]]]	None	Metadata to attach to the produced documents.

Related Information

• TwelveLabsTextEmbedder
• TwelveLabsDocumentEmbedder
• Pipeline Components