What's a Relevance Score

A relevance score is a metric used to evaluate how well the results that the model returns match the query. It's a numerical value that represents the degree to which the information retrieved is applicable or pertinent to the user's request.

Relevance scores are calculated by a model in a pipeline and then assigned to the search result. In extractive question answering pipelines, the score is calculated by the Reader model. In document retrieval pipelines, the score is calculated by the Retriever or by the Ranker model if the pipeline uses it.

What's It For?

Relevance scores tell you how correct the answers are. If a model has a relevance score of 90%, you can expect that its predictions are correct in 9 out of 10 cases.

Relevance scores are particularly useful if you need your search results to meet a particular accuracy threshold.

Data Alignment

If the data the model was trained on strongly differs from the data used for the search, the relevance score may not reflect the model's accuracy. You must fine-tune the model on a specific dataset in such a case.