• Level: Intermediate
• Time to complete: 10 minutes
• Prerequisites:
  • This tutorial assumes some basic knowledge of deepset Cloud. It also assumes that you have a created and deployed pipeline ready. If you don't, you can complete the Tutorial: Building Your First Question Answering App, Tutorial: Building Your First Document Retrieval App, or Tutorial: Building a Summarization System with a Large Language Model.
  • You must be an Admin to complete this tutorial.
• Goal: After completing this tutorial, you will have demonstrated your pipeline to other users and collected their feedback.

Prepare for Inviting Users

It's important to always get feedback from real users as it gives you an idea of your pipeline performance. Before other users can test your pipeline, you must make them aware of a couple of things:

• Make sure your users understand that if they ask for information about documents that don't exist in deepset Cloud, the system won't be able to find an answer.
• Make sure the users understand that the information they're looking for must exist in the text data stored in deepset Cloud. The application cannot make up answers by itself.
• Explain what queries work well (for example, natural language questions instead of simple keywords or copy-pasted error messages).
• Make your users aware of what data is indexed in their deepset Cloud search application.

Result: The users know how to use your pipeline and what to expect. They're ready to run a search with your pipeline and provide feedback.

Share Your Pipeline Prototype

Users don't need a deepset Cloud account to try your pipeline. You generate a link to the prototype of your pipeline. This link gives them access to the search page to run searches with your pipeline prototype.

1. Log in to deepset Cloud.
2. Click Pipelines.
3. Find a deployed pipeline you want to share, and click Share next to it.
   
4. In the Share as prototype window:
   1. Select the link expiration date. When the link expires, people can no longer access the pipeline prototype.
   2. Add the pipeline description. This can be guidelines, things you want users to pay attention to, and so on. The description will be visible to the users on the Search page.
   3. If needed, enable the metadata filters.
      When you enable this option, the metadata from your files are used as search filters.
   4. Choose if you want users to be able to view the source documents.
   5. Generate and copy the link.
   6. Close the window.
5. Share the link with the people you think should test your pipeline.

Result: You have shared your pipeline prototype with real users. They can now run a search and give feedback in deepset Cloud.

Ask Your Users to Run a Search and Give Feedback

Explain to your users the steps to run a search:

1. Open the link they got from you.
   They land on the Search page without access to other pages in deepset Cloud.
2. Ask a question and click Search.
3. Click the thumbs-up or thumbs-down icon below the answers to indicate if the answers are correct.
   

Result: You have explained to your users how to run a search and evaluate the answers. They can now start giving you feedback.

Analyze User Feedback

The feedback is displayed on the Pipeline Details page. You can also export it to a CSV file.

1. In deepset Cloud, go to Pipelines and click the name of the pipeline you shared. This opens the Pipeline Details page.
2. Scroll to the Detailed Feedback section. Here are all the feedback items your pipeline got. You can filter feedback items by correct document and correct answer. You can also check which user added it.
   

👍

Tip

To make feedback analysis easier, you can tag the feedback item. Just click the query in the Detailed Feedback table to add a tag to it. You can then filter feedback by tags.

Result: Congratulations! You have demonstrated your pipeline to real users and collected their feedback. You can now use it to tweak your pipeline and achieve the best performance.