Review Your Experiment

Check the details of your experiment run, such as the status, the metrics calculated, the predictions your pipeline returned, and more.

📘

You must be an Admin to perform this task.

About This Task

You can review the following information about your experiment:

  • The experiment status. If the experiment failed, check the Debug section to see what went wrong.
  • The details of the experiment: the pipeline and evaluation set used.
  • Metrics for pipeline components. You can see both metrics for integrated and isolated evaluation. For more information about metrics, see Experiment Metrics.
  • The pipeline parameters and configuration used for this experiment. It may be different from the actual pipeline as you can update your pipeline just for an experiment run, without modifying the actual pipeline.
    You can't edit your pipeline in this view.
  • Detailed predictions. Here you can see how your pipeline actually did and what answers it returned (predicted answer) in comparison to the expected answers. For each predicted answer, deepset Cloud displays exact match, F1 score, and a rank. The predictions are shown for each node separately.
    You can export these data into a CSV file. Open the node whose predicitons you want to export and click Download CSV.

Review an Experiment from the UI

  1. Log in to deepset Cloud and go to Experiments.
  2. Click the name of the experiment whose details you want to see. The Experiment Details page opens where you can see all the information about your experiment.

Review an Experiment from a Python SDK

Here's the code that displays the results of an experiment run. You can run it in Jupyter Notebooks within deepset Cloud. You need to Generate an API Key first.

# Imports and setup:
import os
from haystack.utils import DeepsetCloudExperiments


# Set the API key and API endpoint:
os.environ["DEEPSET_CLOUD_API_KEY"] = "<your API key>"
os.environ["DEEPSET_CLOUD_API_ENDPOINT"] = "https://api.cloud.deepset.ai/api/v1"

# Get the evaluation run results:
DeepsetCloudExperiments.get_run(workspace="<your_workspace>", eval_run_name="<eval_run_name>")

Review an Experiment with REST API

Use the Get Eval Run API endpoint to display the results of an experiment run. You need to Generate an API Key first.

curl --request GET \
     --url https://api.cloud.deepset.ai/api/v1/workspaces/<WORKSPACE_NAME>/eval_runs/<EVAL_RUN_NAME> \
     --header 'Accept: application/json' \
     --header 'Authorization: Bearer <YOUR_API_KEY>'