HallucinationDetector solves the problem of models making up answers by highlighting the answers that don't exist in the context.


Changes on their way

We're still actively working on this feature to make it better. This page describes its current, first implementation. We'll be updating it soon to make it smoother.

HallucinationDetector is designed for retrieval-augmented generative question answering systems, where the model generates answers based on your documents. To avoid hallucinations, HallucinationDetector uses a model to indicate the answers that are grounded in the documents and the ones that are not.

If you add HallucinationDetector to your generative pipeline, you can then enable it on the Search page. When enabled. HallucinationDetector highlights each sentence in the answer, indicating if it's grounded in the documents. Green highlight means the sentence is based on the documents. Orange highlight means it's partially based on the documents. Red highlight means it's a hallucination.

An example of an answer with hallucinations underlined An answer to the query "how do you take care of a cherry tree" with some sentences underlined in green and some in orange.

Basic Information

  • Pipeline type: Used in query pipelines.
  • Position in a pipeline: After PromptNode.
  • Input: Documents
  • Output: Answers
  • Available Classes: TransformersHallucinationDetector

Usage Example

It's best to leave the node with its default parameters:

	- name: HallucinationDetector
    type: TransformersHallucinationDetector
  - name: query
      - name: Retriever
        inputs: [Query]
      - name: PromptNode
        inputs: [Retriever]
      - name: HallucinationDetector
        inputs: [PromptNode]


ArgumentTypePossible ValuesDescription
model_name_or_pathStringIt uses a model we trained for detecting hallucinations.The model used for detecting whether an answer is grounded in the documents.
model_versionStringThe version of the model.
languageStringcs - Czech
da - Danish
de - German
el - Greek
en - English
es - Spanish
et - Estonian
fi - Finnish
fr - French
it - Italian
ml - Malayam
nl - Dutch
no - Norwegian
pl - Polish
pt - Portugese
ru - Russian
sl - Slovene
sv - Swedish
tr - Turkish

Default: en
The language of the documents using the nltk format.
Default: True
Uses GPU if available. If not, it falls back on the CPU.
batch_sizeIntegerDefault: 16The batch size for inference.
use_auth_tokenString, BooleanAn API token if you're using a private model.
devicesString, torch.deviceA list of devices to use for inference.
class_thresholdsString, floatDefault: "full_support": 1.5
"partial_support": -0.5
"no_support": -0.2
The thresholds for the classes.
The classes are:
- full_support (the whole sentence is grounded in the documents)
- partial_support (a part of the sentence is grounded in the documents)
- no_support (the sentence is not in the documents)
- contradiction (the sentence contradicts the documents).
We trained the model to return:
2 for full_support
1 for partial_support
0 for no_support
-1 for contradiction.
Thresholds are inclusive, which means that if full_support is 1.3, the value of 1.3 is considered full_support.
Changing the thresholds between the classes can help you calibrate what the model considers as grounded in the documents or not.
Default: contradiction
The default class if the value is below all thresholds.