VLLMDocumentEmbedder

Compute embeddings for a list of documents using embedding models served with vLLM. Use this component in indexing pipelines to prepare documents for embedding-based retrieval.

Key Features

• Works with any embedding model served by a vLLM server using the OpenAI-compatible Embeddings API.
• Stores the computed embedding in each document's embedding field.
• Supports adding metadata fields to the text before embedding.
• Configurable batch size for efficient processing.
• Supports vLLM-specific parameters through extra_parameters.

Embedding Models in Query Pipelines and Indexes

The embedding model you use to embed documents in your indexing pipeline must be the same as the embedding model you use to embed the query in your query pipeline.

This means the embedders for your indexing and query pipelines must match. For example, if you use CohereDocumentEmbedder to embed your documents, you should use CohereTextEmbedder with the same model to embed your queries.

Configuration

1. Drag the VLLMDocumentEmbedder component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   1. Set the model to the embedding model served by your vLLM instance.
   2. Set the api_base_url to your vLLM server address. The default is http://localhost:8000/v1.
4. Go to the Advanced tab to configure batch_size, meta_fields_to_embed, embedding_separator, prefix, and suffix.

Connections

VLLMDocumentEmbedder receives a list of documents, typically from a document splitter or converter. It outputs the same documents with embeddings added to their embedding field, ready to be sent to a DocumentWriter.

Source Code

To check this component's source code, open document_embedder.py in the Haystack Core Integrations repository.

Usage Examples

Basic Configuration

  VLLMDocumentEmbedder:
    type: haystack_integrations.components.embedders.vllm.VLLMDocumentEmbedder
    init_parameters:
      model: BAAI/bge-large-en-v1.5
      api_base_url: http://localhost:8000/v1
      batch_size: 32

Using the Component in a Pipeline

# haystack-pipeline
components:
  VLLMDocumentEmbedder:
    type: haystack_integrations.components.embedders.vllm.VLLMDocumentEmbedder
    init_parameters:
      model: BAAI/bge-large-en-v1.5
      api_base_url: http://localhost:8000/v1
      batch_size: 32
      meta_fields_to_embed:
      embedding_separator: "\n"

  document_writer:
    type: haystack.components.writers.document_writer.DocumentWriter
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          index: my-index
          embedding_dim: 1024

connections:
- sender: VLLMDocumentEmbedder.documents
  receiver: document_writer.documents

max_runs_per_component: 100

metadata: {}

Parameters

Inputs

Parameter	Type	Description
documents	List[Document]	A list of documents to embed.

Outputs

Parameter	Type	Description
documents	List[Document]	The documents with their embedding field populated.
meta	Dict[str, Any]	Metadata about the embedding request.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
model	str	(required)	The name of the embedding model served by the vLLM instance.
api_key	Optional[Secret]	Secret.from_env_var("VLLM_API_KEY", strict=False)	An API key for authenticated vLLM deployments.
api_base_url	str	http://localhost:8000/v1	The URL of the vLLM server's OpenAI-compatible API.
prefix	str	""	A string to add at the beginning of each document's text before embedding.
suffix	str	""	A string to add at the end of each document's text before embedding.
dimensions	Optional[int]	None	The number of dimensions in the output embedding.
batch_size	int	32	The number of documents to process in each batch.
progress_bar	bool	True	Whether to show a progress bar during embedding.
meta_fields_to_embed	Optional[List[str]]	None	A list of document metadata field names to include in the text before embedding.
embedding_separator	str	"\n"	The separator used to join the document text and metadata fields.
timeout	Optional[float]	None	Request timeout in seconds.
max_retries	Optional[int]	None	Maximum number of retries on API errors.
http_client_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for the HTTP client.
raise_on_failure	bool	False	Whether to raise an exception on individual document embedding failures.
extra_parameters	Optional[Dict[str, Any]]	None	Additional vLLM-specific parameters for the embedding request.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
documents	List[Document]		A list of documents to embed.

Related Information

• Pipeline Components