Basic Search Functionality

Make sure the users know what to expect from your system. For example:

• For extractive question answering systems that highlight the answer in the text, make sure the users understand that the information they're looking for must exist in the text data the system runs on. The search system cannot make up answers by itself.
• For generative question answering pipelines that aren't chat, make sure the users realize it's not chat-based. The system generates novel answers, but they're unrelated to the previous question.
• What the system returns depends on the pipeline you defined. For example, for a Retriever-only pipeline, the system returns text passages that fit the query, while for a question answering pipeline, the system also highlights the answers within the text passages it returns. Make sure your users know this.
• Explain what queries work well (for example, natural language questions as opposed to simple keywords or copy-pasted error messages).

Data

• Make your users aware of what data is indexed in their deepset Cloud search application. This is the data the system searches. For example, if your system runs on an internal company database, it can answer questions about information in this database. It won't work for questions from other domains.
• Make sure your users understand that if they ask for information about documents that don't exist in deepset Cloud, the system won't be able to find an answer.

Relevance Scores

• Extractive question answering and document retrieval systems add relevance scores to answers. Make sure your users understand the relevance scores displayed beneath each model prediction. You can ask them to read Relevance Scores.
• Set the right threshold for the relevance scores if some users might be confused by model predictions with very low relevance.

Feedback

• To collect users' feedback, ask them to click thumbs up or thumbs down for each answer. User feedback helps in evaluating and improving your pipeline. It lets you see the types of questions the model struggles with. See Collect User Feedback for more details.

✅

Tip

We recommend that you instruct the users that they shouldn't be too strict about the results. If a result helps to answer their question, ask them to select the thumbs-up icon. This includes answers that, for example, lack a word or have the whole sentence highlighted even though just a part of it would be enough.

If a result is garbage text, completely false, or not helpful at all, they should use the thumbs-down icon.