Upload an Evaluation Set
You need an evaluation set to run an experiment. An evaluation dataset is a sample of data that your model never saw during the training. It contains gold answers which are then compared against the actual answers that your pipeline retrieved during evaluation.
Info
- You must be an Admin to perform this task.
- Evaluation sets must be UTF-8-encoded CSV files with comma-separated values.
Prerequisites
Prepare an evaluation dataset for your pipeline type. Evaluation datasets are different for question answering, document search, and RAG.
Upload an Evaluation Dataset from the UI
- Log in to deepset Cloud and go to Evaluation Sets>Upload Evaluation Sets.
- Drop your file into the Upload Evaluation Sets window and click Upload.
- Wait for the upload to finish.
Upload an Evaluation Dataset with REST API
Here's the code to use. You need to Generate an API Key first.
Follow the step-by-step instructions:
▶️
Import Evaluation Dataset
Open Recipe
Or copy this code:
curl --request POST \
--url https://api.cloud.deepset.ai/api/v1/workspaces/<YOUR_WORKSPACE>/evaluation_sets/import \
--header 'Accept: application/json' \
--header 'Authorization: Bearer <YOUR_API_KEY>' \
--header 'Content-Type: multipart/form-data' \
--form 'file=@FILE_NAME.csv'
For more information about this endpoint, see Import Evaluation Set API.
Updated 5 months ago