Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Read an AI Operator RunBeta

Beta
This is coming — this page describes a feature that's still being rolled out and may not be available to you yet.

Check where a measurement or optimization run stands, why it failed or stopped, and which conversations it used.

About This Task​

The Operator page shows the pipeline name and the version of the live revision under the Service name, for example my-pipeline · v295. To see every Operator in the workspace, go to Operate > AI Operator. Each row shows the Service, the goal written as a sentence, a status read from the latest measurement run, and when the Operator last measured the Service. Click a row to open that Operator's page.

Follow a Run​

  1. Open the Operator page for the Service.
  2. Read the header of the run progress card. It names the run's baseline version, for example Baseline: my-pipeline · v295. If the version isn't the one you meant to measure, you can tell as soon as the run starts.
  3. Read the steps of the run. While the run is in progress, each step shows its current state. When the run ends, each step keeps what it did:
    • Sessions replayed, which you can still click to open.
    • Configurations proposed.
    • Configurations tested.
    • Total run time.

If a run fails or you stop it, the card keeps its steps and marks the step where the run ended. The reason appears under that step, not in a separate banner. For example, Failed here. The optimizer returned no candidates. appears in red under the Propose configurations step. A run you stopped shows Stopped here before it finished. in amber.

Check What a Run Measured​

  1. Open the run on the Operator page.
  2. Click the header of the Conversations in this run section to expand it. Even when it's collapsed, the header shows:
    • The number of conversations.
    • The pipeline version the run measured.
    • How the conversations were chosen: sampled from the Service's traffic, or taken from a dataset. For a dataset run, the header gives the dataset's name. If the dataset was deleted or its name is still loading, the header says that the run took a dataset.
  3. If a run measured only a few conversations, an amber Small sample tag appears in the header. Hover over the tag to read the explanation. Results of a small sample are less reliable.

View Past Runs​

To look back at an earlier run, use the Runs table on the Operator page. It lists every run of the Service's Operator, newest first, and appears once the Service has two or more runs.

  1. Find the run in the table. For each run, it shows:
    • Started: when the run started. Hover over the time to see the exact date. The newest run has a Latest tag, and the run currently shown on the page has a Viewing tag.
    • Version: the pipeline version the run measured. If the version isn't known, the table shows a placeholder.
    • Conversations: which conversations the run measured: sampled, hand-picked, or from a dataset. For a dataset run, the column shows the dataset's name.
    • Tested: how many configurations the run tested out of the maximum, for example "2 of 3 configurations".
    • Result: Running, Found an improvement, No improvement, Inconclusive, Stopped, or Failed.
  2. Click a run to open it. The table shows 10 runs at first. To load 10 more, click Show older runs.

You can accept or decline suggestions only for the latest run. Suggestions from earlier runs are read-only.