Start with Suggested Evaluation GoalsBeta
Get a first Evaluation goal for a pipeline without working out the Evaluator, threshold, and pass rate yourself. The AI Assistant suggests a goal and fills in the form, and you review it before you save.
About This Task
An Evaluation goal says when a session passes on a Metric and how many sessions must pass for the goal to be met. You create a goal only through New Evaluation goal. Evaluation goals aren't added to the page as drafts, and nothing is stored until you click Save in the form.
The AI Assistant opens next to the form and suggests a goal in the conversation. When a reply holds exactly one suggestion, the form fills in on its own once the reply is finished. When a reply holds several suggestions, the form stays as it is, and you choose which suggestion to use. If a suggestion names an existing Evaluator, the form copies that Evaluator's code. If the Evaluator is a new judge, the form shows its code so you can review it before you save.
To create a goal, click New Evaluation goal and fill in the form yourself. For the steps, see Define Evaluation Goals on a Pipeline.
Create a Goal with the AI Assistant
- Open your pipeline and click Evaluation.
- Click New Evaluation goal. The goal form opens, and the AI Assistant starts a new conversation beside it. The assistant prepares its first suggestion on its own, which takes a few seconds. While it works, a notice on the form and a message in the assistant panel say it is getting ready, so you can wait for the suggestion before you start filling in the form.
- Describe what you want to check, or let the assistant guide you. Each suggestion appears in the conversation as a card with the goal's name and pass rule. A sole suggestion fills in the form when the reply is finished. If you click Use this on its card before then, the form fills in right away and isn't filled in again over your edits.
- If a reply holds several suggestions, click Use this on the card you want to fill in the form. You can also click Use this on an earlier suggestion in the same chat to fill in the form again.
- Review the name, Evaluator, pass rule, and minimum pass rate. Change anything you want, or ask the assistant for another suggestion.
- Click Save. The platform creates the Evaluator, and then the goal.
- To check the goal on your live sessions, click Evaluate now. Scoring runs in the background. When it finishes, the goal shows whether it is met. For details, see Evaluating Step-by-Step.
For all the fields of the form, see Define Evaluation Goals on a Pipeline.
Refine a Saved Goal with the AI Assistant
To change a goal you already saved, open it and click Refine with AI. When the assistant suggests a revision, it shows the changes next to your current goal. Click Accept to apply the revision, and confirm when asked. Click Discard to keep the goal as it is.
Troubleshooting
- A suggestion appears as a read-only card and doesn't fill in anything. No create form is open, so the assistant can't fill one in. Nothing is stored. Click New Evaluation goal to create the goal with the assistant. After you open the form, click Use this on the card to fill it in.
- A suggestion in a restored chat doesn't fill in the form. Suggestions from an earlier chat only fill in the form when you click Use this on the card while a create form is open.
- The form didn't fill in on its own. The reply holds several suggestions or a summary card for creating several goals, so no single suggestion is chosen for you. Click Use this on the suggestion you want.
- The form is filled in without a Metric. The assistant's suggestion uses code that the form can't read as a Metric declaration, or the Evaluator list couldn't be loaded. Check the Evaluator code, and set the pass rule yourself after the form reads the Metric.
Related Information
Was this page helpful?