Basic Information

Pipeline type: Query
Type: deepset_cloud_custom_nodes.augmenters.reference_predictor.ReferencePredictor
Components it can connect with:
- Generators: A Generator sends the generated replies to ReferencePredictor that adds predicted references to each answer.
- AnswerBuilder: AnswerBuilder can receive the answers with references from ReferencePredictor.

Inputs

Name	Type	Description
`answers`	List of GeneratedAnswer objects	The generated answers to which you want to add references.

Outputs

Name	Type	Description
`answers`	List of GeneratedAnswer objects	Generated answers with a metadata field `_references` added to each answer and containing a list of references this answer was based on. Each reference contains the following fields: - `document_start_idx`: The starting position of the reference in the document. - `document_end_idx`: The end position of the reference in the document. - `answer_start_idx`: The starting position of the reference in the answer. - `answer_end_idx`: The end position of the reference in the answer. - `score`: The score expressing the strength of the association between answer span and document reference span. - `document_id`: The ID of the referenced document. - `document_position`: A 1-based index of the referenced document among all documents passed in for prediction. - `label`: Label for the reference.

Overview

ReferencePredictor shows references to documents on which the LLM's answer is based. Pipelines that contain ReferencePredictor return answers with references next to them. You can easily view the reference to check if the answer is based on it and ensure the model didn't hallucinate.

The default ReferencePredictor model only works for English data. For other languages, use DeepsetAnswerBuilder, and in the prompt, instruct the model to add references to the generated answers.

This is what the references look like in the interface:

Recommended Settings

Reference Prediction Model

Reference prediction model is the model ReferencePredictor uses to compare the similarity of sentences between the answer and source documents. The default model is cross-encoder/ms-marco-MiniLM-L-6-v2.

Verifiability Model

You can indicate a model you want to use to verify if the generated answers need verification in the verifiability_model_name_or_path parameter. ReferencePredictor uses the tstadel/answer-classification-setfit-v2-binary model by default. We trained this model to reject answers that are noise, out of context, or information there was no answer found. It was trained on English data and works at the sentence level, meaning it verifies full sentences. If your data is in other languages, either provide your own model or set verifiability_model_name_or_path to null.

Splitting Rules

To make sure answers are split correctly, we recommend applying additional rules to the sentence-splitting tokenizer. To apply the rules, set use_split_rules to True.

Abbreviations

We recommend extending the number of abbreviations Punkt tokenizer detects to ensure better sentence splitting. You can do this by setting extend_abbreviations to True.

Usage Example

This is an example of a query pipeline in which ReferencePredictor sends answers with references to AnswerBuilder:

components:
...
   reference_predictor:
    type: deepset_cloud_custom_nodes.augmenters.reference_predictor.ReferencePredictor
    init_parameters:
      use_split_rules: True
      extend_abbreviations: True
    
    answer_builder:
      type: haystack.components.builders.answer_builder.AnswerBuilder
      init_parameters: {} # In this example, we're using AnswerBuilder with default parameters
        ...
        
connections:
...
- sender: reference_predictor.answers
  receiver: answer_builder.answers

Init Parameters

Parameter	Type	Possible values	Description
`model`	String	Default: `cross-encoder/ms-marco-MiniLM-L-6-v2`	The name identifier of the model from Hugging Face or the path to a local model folder. Required.
`revision`	String	Default: `None`	The revision of the model to be used. Optional.
`max_seq_len`	Integer	Default: `512`	The maximum number of tokens that a sequence should be truncated to before inference. Required.
`language`	Language	Default: `en`	The language of the data that you want to generate references for. Needed to apply the right sentence splitting rules. Required.
`device`	ComponentDevice	Default: `None`	The device on which the model is loaded. If `None`, the default device is automatically selected. Optional.
`batch_size`	Integer	Default: `16`	The batch size that should be used for inference. Required.
`answer_window_size`	Integer	Default: `1`	The number of sentences of an answer that should be included in one span for inference. Required.
`answer_stride`	Integer	Default: `1`	The stride size for answer window. Required.
`document_window_size`	Integer	Default: `3`	The number of sentences of a document that should be included in one span for inference. Required.
`document_stride`	Integer	Default: `3`	The stride size for document window. Required.
`token`	Secret	Default: `Secret.from_env_var("HF_API_TOKEN", strict=False)`	The token to use as HTTP bearer authorization for remote files. Optional.
`function_to_apply`	String	`sigmoid`, `softmax`, `none` Default: `sigmoid`	The activation function to use on top of the logits. Required.
`min_score_2_label_thresholds`	Dictionary	Default: `None`	The minimum prediction score threshold for each corresponding label. Optional.
`label_2_score_map`	Dictionary	Default: `None`	If using a model with a multi label prediction head, pass in a dictionary mapping label names to a float value. Optional.
`reference_threshold`	Integer	Default: `None`	The minimum score threshold to determine if a prediction should be included as reference or not. Optional.
`default_class`	String	Default: `not_grounded`	A fallback class that should be used if the predicted score doesn't match any threshold. Required.
`verifiability_model`	String	Default: `tstadel/answer-classification-setfit-v2-binary`	The name identifier of the verifiability model to be used on the Hugging Face hub or the path to a local model folder. Optional.
`verifiability_revision`	String	Default: `None`	The revision of the verifiability model to be used. Optional.
`verifiability_batch_size`	Integer	Default: `32`	The batch size that should be used for verifiability inference. Required.
`needs_verification_classes`	List of strings	Default: `["needs_verification"]`	The class names to be used to determine if a sentence needs verification. Required.
`use_split_rules`	Boolean	`True`, `False` Default: `False`	If `True`, additional rules for better splitting answers are applied to the sentence splitting tokenizer. Required.
`extend_abbreviations`	Boolean	`True`, `False` Default: `False`	If `True`, the abbreviations used by NLTK's PunktTokenizer are extended by a list of curated abbreviations if available. If `False`, the default abbreviations are used. Required.
`model_kwargs`	Dictionary	Default: `None`	Additional keyword arguments for the model. Optional.
`verifiability_model_kwargs`	Dictionary	Default: `None`	Additional keyword arguments for the verifiability model. Optional.