Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

AI OperatorBeta

Beta
This is coming — this page describes a feature that's still being rolled out and may not be available to you yet.

An AI Operator is an optimization goal set on one Service. Use it when you want to lower what a Service costs without lowering the quality of its answers. It measures what the Service costs and how well it answers, then suggests cheaper configurations for you to review.

What an AI Operator Does​

Each Service can have at most one AI Operator. When you add one, you choose:

  • The running Service you want to optimize.
  • The Experiment whose evaluators judge the quality of that Service's answers.
  • The quality the Service must keep while being optimized.

The AI Operator then measures a baseline for the Service: its cost, latency, and quality on the pipeline version you choose, based on a set of conversations. Once it has a baseline, you can ask it for optimization suggestions. The AI Operator proposes alternative, usually cheaper, pipeline configurations and validates that they still meet the quality you set.

Suggestions are advisory only. The AI Operator does not apply or promote configurations by itself — you review what it finds and decide what to do with it.

Where You Find It​

The AI Operator has two entry points:

  • Operator list (go to Operate > AI Operator): to check on all optimization goals in the workspace, use this list. It shows each goal's Service, the goal written as a sentence, a status read from the latest measurement run, and when it last ran. Click a row to open that Operator.
  • Service page: to set up or open the AI Operator for a specific Service, use the Service page. A Service with no goal shows a Set up AI Operator button next to the actions menu. A Service that already has a goal shows an AI Operator button instead, which opens its Operator page directly.

Key Ideas​

  • One goal per Service: you can't add a second AI Operator to a Service that already has one. The existing Service still appears in the Add AI Operator list, but it's disabled.
  • Experiment-backed quality: the quality threshold the Operator must keep comes from the evaluators defined in an Experiment. A pipeline with no Experiment can't get an AI Operator until you create one.
  • Baseline before suggestions: before the AI Operator can propose anything, it needs a baseline measurement run on a chosen pipeline version using a set of conversations, split between training and held-out sets. If some Sessions can't be replayed or too few replayable Sessions remain, the AI Operator refuses the run. See Fix a Refused Run.