deepset Cloud uses readers that are:

• Built on the latest transformer-based language models
• Strong in their grasp of semantics
• Sensitive to syntactic structure
• State-of-the-art in question-answering (QA) tasks like SQuAD and Natural Questions

Our readers contain all the components of end-to-end, open-domain QA systems, including:

• Loading of model weights
• Tokenization
• Embedding computation
• Span prediction
• Candidate aggregation

If you use a reader in your pipeline, it highlights phrases and sentences as answers to your query.

Usage

Readers are usually combined with retrievers in pipelines. To define a reader:

#Import the reader:
from haystack.nodes import FARMReader

#Specify the model that you want to use with your reader:
model = "deepset/roberta-base-squad2"

#Specify the reader:
reader = FARMReader(model, use_gpu=True)

Or in YAML:

components:
    -name: MyReader
  type: FARMReader
  params:
    model: "deepset/roberta-base-squad2"
    use_gpu: True

Models

A reader always takes a model as an argument. deepset Cloud readers handle loading model weights so to use a pre-trained QA model with your reader, simply provide its Hugging Face model hub name.

There are plenty of models out there and it can be difficult to select one. Here are some tips to help you get started:

• RoBERTa (base): An optimized variant of BERT and a great starting point. Can be handled by any machine with a single NVidia V100 GPU.
  • PRO: Strong all-round model
  • CON: There are faster and more accurate models
  • HUB NAME: deepset/roberta-base-squad1
• MiniLM: A cleverly distilled model that sacrifices some accuracy for speed. Recommended if you prioritize speed and GPU memory over accuracy. Outperforms BERT base on SQuAD.
  • PRO: Inference speed up to 50% faster than BERT base
  • CON: Doesn't match the best base-sized models in accuracy.
  • HUB NAME: deepset/minilm-uncased-squad2
• ALBERT (XXL): A large and powerful SotA model. If you want the best performance and you have the computational resources, this is the model for you.
  • PRO: Better accuracy than any other open-source model in QA.
  • CON: Needs a lot of computational power which makes it impractical in most use cases.
  • HUB NAME: ahotrod/albert_xxlargev1_squad2_512

📘

Models for TableReader

The nq-reader models used with TableReader can provide confidence scores but cannot handle questions that need aggregation over multiple cells. The answers are sorted by a general table score first, and then by answer span scores.

If you want to learn more about models, see Language Models.

Reader Types

TableReader

This reader retrieves answers to your questions even if they are buried in a table. It is designed to use the TAPAS model by Google (google/tapas-base-finetuned-wtq). This model can return a single cell as an answer or can pick a set of cells and then aggregate them to get the final answer. It uses the Hugging Face transformers framework.
For a full list of models available for this reader, see Hugging Face Models.

Usage

These are the arguments that you can specify for TableReader:

FARMReader

A transformer-based model for extractive QA using the FARM framework. You can only use encoder models with FARM reader, such as BERT, ELECTRA, RoBERTa, ALBERT, XML, DistilBERT, DeBERTa.

Main Features

• Removes duplicates
• Uses the tokenizers from the Hugging Face transformers library
• Start and end logits are summed and not normalized

Usage

These are the arguments that you can specify for FARMReader:

TransformersReader

An alternative to the FARMReader that directly uses the Transformers library. It has less features than the FARMReader and so we only recommend it if you would like to bypass FARM. For a comparison of the two Readers, see FARM vs Transformers.

Usage

These are the arguments that you can specify for TransformersReader: