DeepsetNvidiaRanker

Rank documents by their relevance to the query using NVIDIA Triton.

Basic Information

• Type: deepset_cloud_custom_nodes.rankers.nvidia.ranker.DeepsetNvidiaRanker
• Components it most often connects to:
  • Retrievers: DeepsetNvidiaRanker can receive documents from a Retriever and then rank them.
  • PromptBuilder: DeepsetNvidiaRanker can send the ranked documents to PromptBuilder, which adds them to the prompt for the LLM.
  • Any component that outputs a list of documents or accepts a list of documents as input.

Inputs

Parameter	Type	Default	Description
query	str		The input query to compare the documents to.
documents	List[Document]		A list of documents to be ranked.
top_k	int	None	None
scale_score	bool	None	None
calibration_factor	float	None	None
score_threshold	float	None	None

Outputs

Parameter	Type	Default	Description
documents	List[Document]		A list of documents closest to the query, sorted from most similar to least similar.

Overview

DeepsetNvidiaRanker uses the NVIDIA Triton Inference Server to rank documents by their similarity to the query, assigning similarity scores to each document.

This component runs on optimized hardware and is usable within deepset AI Platform only, which means it doesn't work if you export it to a local Python file. If you're planning to export, use TransformersSimilarityRanker instead.

Usage Example

Initializing the Component

components:
  DeepsetNvidiaRanker:
    type: rankers.nvidia.ranker.DeepsetNvidiaRanker
    init_parameters:

Using the Component in a Pipeline

This is an example of a query pipeline where DeepsetNvidiaRanker receives documents from a DocumentJoiner, ranks them, and then returns them as the pipeline output.

Here's the YAML configuration:

components:
  bm25_retriever:
    type: haystack_integrations.components.retrievers.opensearch.bm25_retriever.OpenSearchBM25Retriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          use_ssl: true
          verify_certs: false
          hosts:
            - ${OPENSEARCH_HOST}
          http_auth:
            - ${OPENSEARCH_USER}
            - ${OPENSEARCH_PASSWORD}
          embedding_dim: 768
          similarity: cosine
      top_k: 20
  embedding_retriever:
    type: haystack_integrations.components.retrievers.opensearch.embedding_retriever.OpenSearchEmbeddingRetriever
    init_parameters:
      document_store:
        type: haystack_integrations.document_stores.opensearch.document_store.OpenSearchDocumentStore
        init_parameters:
          use_ssl: true
          verify_certs: false
          hosts:
            - ${OPENSEARCH_HOST}
          http_auth:
            - ${OPENSEARCH_USER}
            - ${OPENSEARCH_PASSWORD}
          embedding_dim: 768
          similarity: cosine
      top_k: 20
  document_joiner:
    type: haystack.components.joiners.document_joiner.DocumentJoiner
    init_parameters:
      join_mode: concatenate
  DeepsetNvidiaTextEmbedder:
    type: deepset_cloud_custom_nodes.embedders.nvidia.text_embedder.DeepsetNvidiaTextEmbedder
    init_parameters:
      model: intfloat/multilingual-e5-base
      prefix: ''
      suffix: ''
      truncate: null
      normalize_embeddings: false
      timeout: null
      backend_kwargs: null
  DeepsetNvidiaRanker:
    type: deepset_cloud_custom_nodes.rankers.nvidia.ranker.DeepsetNvidiaRanker
    init_parameters:
      model: intfloat/simlm-msmarco-reranker
      query_prefix: ''
      document_prefix: ''
      top_k: 10
      score_threshold: null
      meta_fields_to_embed: null
      embedding_separator: \n
      scale_score: true
      calibration_factor: 1
      timeout: null
      backend_kwargs: null
connections:
  - sender: bm25_retriever.documents
    receiver: document_joiner.documents
  - sender: embedding_retriever.documents
    receiver: document_joiner.documents
  - sender: DeepsetNvidiaTextEmbedder.embedding
    receiver: embedding_retriever.query_embedding
  - sender: document_joiner.documents
    receiver: DeepsetNvidiaRanker.documents
max_runs_per_component: 100
metadata: {}
inputs:
  query:
    - bm25_retriever.query
    - DeepsetNvidiaTextEmbedder.text
    - DeepsetNvidiaRanker.query
  filters:
    - bm25_retriever.filters
    - embedding_retriever.filters
outputs:
  documents: DeepsetNvidiaRanker.documents

Parameters

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
model	DeepsetNVIDIARankingModels	DeepsetNVIDIARankingModels.INTFLOAT_SIMLM_MSMARCO_RERANKER	The model to use for ranking. Choose the model from a list on the component card.
query_prefix	str		String to prepend to queries
document_prefix	str		String to prepend to documents
top_k	int	10	Maximum number of documents to return
batch_size	int	40	The number of documents to rank at once.
score_threshold	float	None	None
meta_fields_to_embed	List[str]	None	None
embedding_separator	str	\n	Separator for concatenating metadata fields
scale_score	bool	True	Whether to scale the scores using a sigmoid function.
calibration_factor	float	None	1.0
timeout	float	None	None
backend_kwargs	Dict[str, Any]	None	None

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
query	str		The input query to compare the documents to.
documents	List[Document]		A list of documents to be ranked.
top_k	int | None	None	The maximum number of documents to return.
scale_score	bool | None	None	If True, scales the raw logit predictions using a Sigmoid activation function. If False, disables scaling of the raw logit predictions.
calibration_factor	float | None	None	Use this factor to calibrate probabilities with sigmoid(logits * calibration_factor). Used only if scale_score is True.
score_threshold	float | None	None	The minimum score for documents to be included in the results.