Relevance Scores
When you run a search, each search result has a relevance score that reflects the model's confidence in it. Explore the significance of relevance scores in determining the accuracy of a model's predictions. Learn how they guide achieving desired accuracy thresholds and why aligning your model with the specific data it analyzes is crucial for maintaining reliability.
What's a Relevance Score
A relevance score is a metric used to evaluate how well the results that the model returns match the query. It's a numerical value that represents the degree to which the information retrieved is applicable or pertinent to the user's request.
Relevance scores are calculated by a model in a pipeline and then assigned to the search result. In extractive question answering pipelines, the score is calculated by the Reader model. In document retrieval pipelines, the score is calculated by the Retriever or by the Ranker model if the pipeline uses it.
What's It For?
Relevance scores tell you how correct the answers are. If a model has a relevance score of 90%, you can expect that its predictions are correct in 9 out of 10 cases.
Relevance scores are particularly useful if you need your search results to meet a particular accuracy threshold.
Data Alignment
If the data the model was trained on strongly differs from the data used for the search, the relevance score may not reflect the model's accuracy. You must fine-tune the model on a specific dataset in such a case.
Updated 7 months ago