ReferencePredictor

Use this component in retrieval augmented generation (RAG) pipelines to predict references for the generated answer.

Basic Information

  • Type: deepset_cloud_custom_nodes.augmenters.reference_predictor.ReferencePredictor
  • Components it can connect with:
    • Generators: A Generator sends the generated replies to ReferencePredictor that adds predicted references to each answer.
    • AnswerBuilder: AnswerBuilder can receive the answers with references from ReferencePredictor.

Inputs

NameTypeDescription
answersList of GeneratedAnswer objectsThe generated answers to which you want to add references.

Outputs

Name

Type

Description

answers

List of GeneratedAnswer objects

Generated answers with a metadata field _references added to each answer and containing a list of references this answer was based on. Each reference contains the following fields:

  • document_start_idx: The starting position of the reference in the document.
  • document_end_idx: The end position of the reference in the document.
  • answer_start_idx: The starting position of the reference in the answer.
  • answer_end_idx: The end position of the reference in the answer.
  • score: The score expressing the strength of the association between answer span and document reference span.
  • document_id: The ID of the referenced document.
  • document_position: A 1-based index of the referenced document among all documents passed in for prediction.
  • label: Label for the reference.

Overview

ReferencePredictor shows references to documents on which the LLM's answer is based. Pipelines that contain ReferencePredictor return answers with references next to them. You can easily view the reference to check if the answer is based on it and ensure the model didn't hallucinate.

The default ReferencePredictor model only works for English data. For other languages, use DeepsetAnswerBuilder, and in the prompt, instruct the model to add references to the generated answers.

This is what the references look like in the interface:

Recommended Settings

Reference Prediction Model

Reference prediction model is the model ReferencePredictor uses to compare the similarity of sentences between the answer and source documents. The default model is cross-encoder/ms-marco-MiniLM-L-6-v2.

Verifiability Model

You can indicate a model you want to use to verify if the generated answers need verification in the verifiability_model_name_or_path parameter. ReferencePredictor uses the tstadel/answer-classification-setfit-v2-binary model by default. We trained this model to reject answers that are noise, out of context, or information there was no answer found. It was trained on English data and works at the sentence level, meaning it verifies full sentences. If your data is in other languages, either provide your own model or set verifiability_model_name_or_path to null.

Splitting Rules

To make sure answers are split correctly, we recommend applying additional rules to the sentence-splitting tokenizer. To apply the rules, set use_split_rules to True.

Abbreviations

We recommend extending the number of abbreviations Punkt tokenizer detects to ensure better sentence splitting. You can do this by setting extend_abbreviations to True.

Usage Example

This is an example of a query pipeline in which ReferencePredictor sends answers with references to AnswerBuilder:

components:
...
   reference_predictor:
    type: deepset_cloud_custom_nodes.augmenters.reference_predictor.ReferencePredictor
    init_parameters:
      use_split_rules: True
      extend_abbreviations: True
    
    answer_builder:
      type: haystack.components.builders.answer_builder.AnswerBuilder
      init_parameters: {} # In this example, we're using AnswerBuilder with default parameters
        ...
        
connections:
...
- sender: reference_predictor.answers
  receiver: answer_builder.answers

Parameters

Init Parameters

These are the parameters you can configure in Pipeline Builder:


Parameter

Type

Possible values

Description

model

String

Default: cross-encoder/ms-marco-MiniLM-L-6-v2

The name identifier of the model from Hugging Face or the path to a local model folder.
Required.

revision

String

Default: None

The revision of the model to be used. Optional.

max_seq_len

Integer

Default: 512

The maximum number of tokens that a sequence should be truncated to before inference.
Required.

language

Language

Default: en

The language of the data that you want to generate references for. Needed to apply the right sentence splitting rules.
Required.

device

ComponentDevice

Default: None

The device on which the model is loaded. If None, the default device is automatically selected.
Optional.

batch_size

Integer

Default: 16

The batch size that should be used for inference.
Required.

answer_window_size

Integer

Default: 1

The number of sentences of an answer that should be included in one span for inference.
Required.

answer_stride

Integer

Default: 1

The stride size for answer window. Required.

document_window_size

Integer

Default: 3

The number of sentences of a document that should be included in one span for inference.
Required.

document_stride

Integer

Default: 3

The stride size for document window.
Required.

token

Secret

Default: Secret.from_env_var("HF_API_TOKEN", strict=False)

The token to use as HTTP bearer authorization for remote files.
Optional.

function_to_apply

String

sigmoid, softmax, none
Default: sigmoid

The activation function to use on top of the logits.
Required.

min_score_2_label_thresholds

Dictionary

Default: None

The minimum prediction score threshold for each corresponding label.
Optional.

label_2_score_map

Dictionary

Default: None

If using a model with a multi label prediction head, pass in a dictionary mapping label names to a float value.
Optional.

reference_threshold

Integer

Default: None

The minimum score threshold to determine if a prediction should be included as reference or not.
Optional.

default_class

String

Default: not_grounded

A fallback class that should be used if the predicted score doesn't match any threshold.
Required.

verifiability_model

String

Default: tstadel/answer-classification-setfit-v2-binary

The name identifier of the verifiability model to be used on the Hugging Face hub or the path to a local model folder.
Optional.

verifiability_revision

String

Default: None

The revision of the verifiability model to be used.
Optional.

verifiability_batch_size

Integer

Default: 32

The batch size that should be used for verifiability inference.
Required.

needs_verification_classes

List of strings

Default: ["needs_verification"]

The class names to be used to determine if a sentence needs verification.
Required.

use_split_rules

Boolean

True, False
Default: False

If True, additional rules for better splitting answers are applied to the sentence splitting tokenizer.
Required.

extend_abbreviations

Boolean

True, False
Default: False

If True, the abbreviations used by NLTK's PunktTokenizer are extended by a list of curated abbreviations if available. If False, the default abbreviations are used. Required.

model_kwargs

Dictionary

Default: None

Additional keyword arguments for the model.
Optional.

verifiability_model_kwargs

Dictionary

Default: None

Additional keyword arguments for the verifiability model.
Optional.

Run Method Parameters

These are the parameters you can configure for the component's run() method. This means you can pass these parameters at query time through the API, in Playground, or when running a job. For details, see Modify Pipeline Parameters at Query Time.

Run() method parameters take precedence over initialization parameters.


Parameter

Type

Description

answers

List of GeneratedAnswer objects

Replies returned by the Generator.
Required.