SentenceTransformersDiversityRanker

Rank documents to maximize diversity relative to the query using Sentence Transformers embeddings. Use this component after retrieval when you want results that cover different aspects of a topic instead of near-duplicate passages.

Key Features

• Supports greedy diversity ordering and maximum margin relevance (MMR) strategies.
• Uses a Sentence Transformers model to embed the query and documents.
• Deduplicates documents by ID before ranking, keeping the highest-scored duplicate when scores are present.
• Supports cosine and dot product similarity metrics.
• Supports prefix and suffix strings and metadata fields to embed with document content.

Configuration

1. Drag the SentenceTransformersDiversityRanker component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   1. Set the model to a Sentence Transformers model (for example, sentence-transformers/all-MiniLM-L6-v2).
   2. Set top_k to control how many documents to return.
   3. Choose a strategy: greedy_diversity_order or maximum_margin_relevance.
4. Go to the Advanced tab to configure similarity, lambda_threshold, prefixes, suffixes, and backend options.

Connections

SentenceTransformersDiversityRanker accepts a query string and a list of documents as inputs. Connect it after a retriever or DocumentJoiner in a query pipeline. It outputs a re-ranked list of documents through its documents output.

Source Code

To check this component's source code, open sentence_transformers_diversity.py in the Haystack Core Integrations repository.

Usage Examples

Basic Configuration

  SentenceTransformersDiversityRanker:
    type: haystack_integrations.components.rankers.sentence_transformers.SentenceTransformersDiversityRanker
    init_parameters:
      model: sentence-transformers/all-MiniLM-L6-v2
      top_k: 5
      strategy: greedy_diversity_order
      similarity: cosine

Using the Component in a Pipeline

# haystack-pipeline
components:
  retriever:
    type: haystack.components.retrievers.in_memory.embedding_retriever.InMemoryEmbeddingRetriever
    init_parameters:
      document_store:
        type: haystack.document_stores.in_memory.document_store.InMemoryDocumentStore
        init_parameters: {}
      top_k: 20

  diversity_ranker:
    type: haystack_integrations.components.rankers.sentence_transformers.SentenceTransformersDiversityRanker
    init_parameters:
      model: sentence-transformers/all-MiniLM-L6-v2
      top_k: 5
      strategy: maximum_margin_relevance
      lambda_threshold: 0.5

connections:
- sender: retriever.documents
  receiver: diversity_ranker.documents

max_runs_per_component: 100

metadata: {}

inputs:
  query:
  - diversity_ranker.query

outputs:
  documents: diversity_ranker.documents

Parameters

Inputs

Parameter	Type	Description
query	str	The search query.
documents	List[Document]	Documents to rank for diversity.
top_k	Optional[int]	Overrides the init-time top_k for this call.
lambda_threshold	Optional[float]	Overrides the init-time trade-off between relevance and diversity when using the maximum_margin_relevance strategy. Must be between 0 and 1.

Outputs

Parameter	Type	Description
documents	List[Document]	Documents selected based on the diversity ranking strategy.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
model	str	sentence-transformers/all-MiniLM-L6-v2	Local path or Hugging Face model name.
top_k	int	10	The maximum number of documents to return per query.
device	Optional[ComponentDevice]	None	The device on which the model is loaded.
token	Optional[Secret]	Secret.from_env_var(["HF_API_TOKEN", "HF_TOKEN"], strict=False)	The API token to download private models from Hugging Face.
similarity	str	cosine	Similarity metric for comparing embeddings. One of dot_product or cosine.
query_prefix	str	""	A string to add to the beginning of the query text before ranking.
query_suffix	str	""	A string to add to the end of the query text before ranking.
document_prefix	str	""	A string to add to the beginning of each document text before ranking.
document_suffix	str	""	A string to add to the end of each document text before ranking.
meta_fields_to_embed	Optional[List[str]]	None	Metadata fields to embed along with the document content.
embedding_separator	str	\n	Separator used to concatenate metadata fields to the document text.
strategy	str	greedy_diversity_order	The diversity ranking strategy. One of greedy_diversity_order or maximum_margin_relevance.
lambda_threshold	float	0.5	Trade-off between relevance and diversity for MMR. Values closer to 0 favor diversity; values closer to 1 favor relevance.
model_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for the model constructor.
tokenizer_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for the tokenizer.
config_kwargs	Optional[Dict[str, Any]]	None	Additional keyword arguments for the model configuration.
backend	Literal["torch", "onnx", "openvino"]	torch	The backend to use for the Sentence Transformers model.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
query	str		The search query.
documents	List[Document]		Documents to rank for diversity.
top_k	Optional[int]	None	Maximum number of documents to return.
lambda_threshold	Optional[float]	None	MMR trade-off parameter override.

Related Information

• SentenceTransformersSimilarityRanker
• Pipeline Components