VLLMRanker

Rank documents by their similarity to a query using reranking models served with vLLM.

Key Features

• Reranks documents based on semantic similarity to the query using the vLLM /rerank endpoint.
• Works with any reranking model served by a vLLM server.
• Returns documents sorted from most to least relevant.
• Configurable top_k and score_threshold to control the number and quality of results.
• Can include metadata fields in the document text for ranking.

Configuration

1. Drag the VLLMRanker component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   1. Set the model to the reranking model served by your vLLM instance.
   2. Set the api_base_url to your vLLM server address. The default is http://localhost:8000/v1.
4. Go to the Advanced tab to configure top_k, score_threshold, and meta_fields_to_embed.

Connections

VLLMRanker receives a query string and a list of documents from a retriever. It outputs a reranked list of documents, sorted by relevance to the query.

Source Code

To check this component's source code, open ranker.py in the Haystack Core Integrations repository.

Usage Examples

Basic Configuration

  VLLMRanker:
    type: haystack_integrations.components.rankers.vllm.VLLMRanker
    init_parameters:
      model: BAAI/bge-reranker-v2-m3
      api_base_url: http://localhost:8000/v1
      top_k: 5

Using the Component in a Pipeline

# haystack-pipeline
components:
  retriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      top_k: 20
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          hosts:
          index: my-index

  VLLMRanker:
    type: haystack_integrations.components.rankers.vllm.VLLMRanker
    init_parameters:
      model: BAAI/bge-reranker-v2-m3
      api_base_url: http://localhost:8000/v1
      top_k: 5

connections:
- sender: retriever.documents
  receiver: VLLMRanker.documents

max_runs_per_component: 100

metadata: {}

inputs:
  query:
  - VLLMRanker.query

Parameters

Inputs

Parameter	Type	Description
query	str	The query to rank documents against.
documents	List[Document]	A list of documents to rerank.
top_k	Optional[int]	The maximum number of documents to return. Overrides the init-time top_k.
score_threshold	Optional[float]	Minimum score threshold. Overrides the init-time score_threshold.

Outputs

Parameter	Type	Description
documents	List[Document]	The reranked documents, sorted from most to least relevant.
meta	Dict[str, Any]	Metadata about the reranking request.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
model	str	(required)	The name of the reranking model served by the vLLM instance.
api_key	Optional[Secret]	Secret.from_env_var("VLLM_API_KEY", strict=False)	An API key for authenticated vLLM deployments.
api_base_url	str	http://localhost:8000/v1	The URL of the vLLM server's API.
top_k	Optional[int]	None	The maximum number of documents to return.
score_threshold	Optional[float]	None	A minimum score threshold. Documents below this score are filtered out.
meta_fields_to_embed	Optional[List[str]]	None	A list of document metadata field names to include in the text when scoring.
meta_data_separator	str	"\n"	The separator used to join the document text and metadata fields.
http_client_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for the HTTP client.
extra_parameters	Optional[Dict[str, Any]]	None	Additional vLLM-specific parameters for the reranking request.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
query	str		The query to rank documents against.
documents	List[Document]		A list of documents to rerank.
top_k	Optional[int]	None	The maximum number of documents to return.
score_threshold	Optional[float]	None	Minimum score threshold for results.

Related Information

• Pipeline Components