Upload an Evaluation Set

You need an evaluation set to run an experiment. An evaluation dataset is a sample of data that your model never saw during the training. It contains gold answers which are then compared against the actual answers that your pipeline retrieved during evaluation.

πŸ“˜

You must be an Admin to perform this task.

Prerequisites

Prepare an evaluation dataset for your pipeline type. Evaluation datasets are different for question answering, document search, and RAG.

Upload an Evaluation Dataset from the UI

  1. Log in to deepset Cloud and go to Evaluation Sets>Upload Evaluation Sets.
  2. Drop your file into the Upload Evaluation Sets window and click Upload.
  3. Wait for the upload to finish.

Upload an Evaluation Dataset with REST API

Here's the code to use. You need to Generate an API Key first.

Follow the step-by-step instructions:

Or copy this code:

curl --request POST \
       --url https://api.cloud.deepset.ai/api/v1/workspaces/<YOUR_WORKSPACE>/evaluation_sets/import \
     --header 'Accept: application/json' \
     --header 'Authorization: Bearer <YOUR_API_KEY>' \
     --header 'Content-Type: multipart/form-data' \
     --form 'file=@FILE_NAME.csv'

For more information about this endpoint, see Import Evaluation Set API.