Ask an AI Operator for New SuggestionsBeta
If a run ends without a suggestion you can use, or the suggestions don't fit your goal, ask the AI Operator for another round on the same run.
About This Task
A run can end without a suggestion to present: it found no improvement, the result was inconclusive, or it failed while proposing or testing configurations. Most of a run's time and cost go into picking and replaying conversations and measuring the baseline. Suggest again reuses all of that: the run keeps its conversations, its training and held-out split, its baseline replays, and its measured baseline. Only proposing and testing new candidates run again, under a fresh deadline. To learn how rounds work, see AI Operator Run Time and Cost.
Suggest again and Run again do different things:
- Suggest again reopens the latest run and tests only new candidates.
- Run again starts a new run from the beginning, including a new baseline measurement. Use it to measure the Service again, for example after you publish a new pipeline version.
The Suggest again button appears next to Run again only when the Service's latest run meets all of these conditions:
- It has ended or failed. It can have ended with no improvement, an inconclusive result, a failure while proposing or testing, or a presented candidate.
- It measured a baseline quality. If it didn't, use Run again.
- None of its candidates is being promoted or was accepted. Once an accept starts, the Service no longer runs the baseline.
It's also hidden on past runs. If the run doesn't meet the conditions, the request is refused and the run stays as it was.
Suggest Again
- On the Operator page, open the Service's latest run. In the Proposed pipeline configurations section, click Suggest again. Hover over the button to see what the run reuses.
- Wait while the Operator proposes and tests new suggestions. A success notification appears, the run reopens at the proposing step, and the page refreshes the run's progress on its own. The run's status shows it as in progress until the new candidates are tested.
- Review the candidates of the latest round together with the measured candidates from earlier rounds. Each candidate shows the proposal round it came from.
Result: The run reopens with a fresh time limit. Candidates from earlier attempts stay on the run, so new candidates are ranked together with them. To review them, see Review AI Operator Suggestions.
Read Suggestions by Round
While the AI Operator proposes a new round, the Proposed pipeline configurations section shows a "generating" indicator, and every suggestion already on the page is folded under Earlier suggestions. When the round finishes, the section shows the latest round of suggestions. Two things can appear next to it:
- The run's recommendation: if a configuration from an earlier round is still the best one, it stays visible next to the latest round, so you don't have to open Earlier suggestions to find it.
- Earlier suggestions (N): a collapsed list of the other earlier rounds, where N is how many suggestions it holds. Expand it to review them. Suggestions in this list are still ranked against new suggestions.
What to Expect
- The AI Operator tries something new. It knows which techniques earlier candidates used, how each one ended, and why you or the AI Operator declined it. A proposal that repeats the configuration of an earlier candidate is dropped before it is measured.
- A presented candidate has to win again. It goes back to being measured and is ranked together with the new round's candidates.
- An empty round doesn't lose your results. If a round produces no candidate that can be measured, the run ranks the measured candidates from earlier rounds instead of failing.
To reopen a run through the API, see Work with AI Operator Suggestions Through the API.
Related Information
Was this page helpful?