HuggingFaceTEIRanker

Ranks documents based on their semantic similarity to the query.

Work in Progress

Bear with us while we're working on adding pipeline examples and most common components connections.

Key Features

• Ranks documents by semantic similarity to the query using a Text Embeddings Inference (TEI) API endpoint
• Works with self-hosted TEI services and Hugging Face Inference Endpoints
• Configurable number of returned documents with top_k
• Built-in retry logic with configurable status codes
• Optional truncation direction support

Configuration

1. Drag the HuggingFaceTEIRanker component onto the canvas from the Component Library.
2. Click on the component to open the configuration panel.
3. On the General tab:
   1. Enter the base URL of your TEI reranking service (for example, https://api.example.com).
   2. Optionally, enter your Hugging Face API token if your TEI server requires authentication. Check your token in your account settings.
   3. Set top_k to control the maximum number of documents to return.
4. Go to the Advanced tab to configure raw scores, timeout, max retries, and retry status codes.

Connections

HuggingFaceTEIRanker accepts a query (str), a list of Document objects, and an optional top_k override and truncation_direction. It outputs a reranked list of Document objects.

Connect a retriever's documents output and the query from Input to this component. Connect this component's documents output to the next stage in your pipeline (for example, a prompt builder or answer builder).

Source Code

To check this component's source code, open hugging_face_tei.py in the Haystack repository.

Usage Examples

Basic Configuration

  HuggingFaceTEIRanker:
    type: components.rankers.hugging_face_tei.HuggingFaceTEIRanker
    init_parameters: {}

In a query pipeline, connect a retriever's documents output to the documents input, and connect the reranked documents output to a prompt builder or answer builder.

components:
  HuggingFaceTEIRanker:
    type: components.rankers.hugging_face_tei.HuggingFaceTEIRanker
    init_parameters:

Parameters

Inputs

Parameter	Type	Default	Description
query	str		The user query string to guide reranking.
documents	List[Document]		List of Document objects to rerank.
top_k	Optional[int]	None	Optional override for the maximum number of documents to return.
truncation_direction	Optional[TruncationDirection]	None	If set, enables text truncation in the specified direction.

Outputs

Parameter	Type	Description
documents	List[Document]	A list of reranked documents.

Init Parameters

These are the parameters you can configure in Pipeline Builder:

Parameter	Type	Default	Description
url	str		Base URL of the TEI reranking service (for example, "https://api.example.com").
top_k	int	10	Maximum number of top documents to return.
raw_scores	bool	False	If True, include raw relevance scores in the API payload.
timeout	Optional[int]	30	Request timeout in seconds.
max_retries	int	3	Maximum number of retry attempts for failed requests.
retry_status_codes	Optional[List[int]]	None	List of HTTP status codes that will trigger a retry. When None, HTTP 408, 418, 429 and 503 will be retried.
token	Optional[Secret]	Secret.from_env_var(['HF_API_TOKEN', 'HF_TOKEN'], strict=False)	The Hugging Face token to use as HTTP bearer authorization. Not always required depending on your TEI server configuration. Check your HF token in your account settings.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Parameter	Type	Default	Description
query	str		The user query string to guide reranking.
documents	List[Document]		List of Document objects to rerank.
top_k	Optional[int]	None	Optional override for the maximum number of documents to return.
truncation_direction	Optional[TruncationDirection]	None	If set, enables text truncation in the specified direction.

Related Information

• Self-hosted Text Embeddings Inference
• Hugging Face Inference Endpoints