Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Deployment GatesBeta

Beta
This is coming — this page describes a feature that's still being rolled out and may not be available to you yet.

A Deployment Gate holds a new service revision until it proves itself. Use a gate when you want to be sure that a new pipeline version scores well before your callers get it. You set up a gate on a service, and it runs each time a new revision is about to go live.

How a Gate Works​

Before a revision goes live, the platform replays a Dataset of test cases through it, scores the results with a Metric Group, and compares the Composite score to a threshold you set. If every check passes, the revision goes live. If a check fails or can't be decided, the previous revision keeps serving requests.

Checks​

A gate is a list of checks. Each check has three parts:

  • Metric Group: The Metric Group to validate. The gate uses the Group's Composite score, so the Group needs weights. The Metric Group list shows only Groups that have at least one usable weight.
  • Dataset: The Dataset of test cases to replay through the new revision.
  • Pass criteria: A comparison (greater than, or greater than or equal to), a threshold for the Composite score, and a minimum coverage. Minimum coverage is the share of test cases that must receive a score for the check to count.

A gate can have one check for each Metric Group. The revision goes live only when all checks pass.

Metric Groups a Gate Accepts​

A gate freezes the Group's weights when you save it, so every weight must be usable. A weight is unusable when its Metric has no Normalized mean. This can happen after you save the Group: if an Evaluator gets a new version that drops the Metric, its range, or its better direction, the weight stops counting.

Saving a gate is refused if its Metric Group weighs any Metric with no Normalized mean, even when other weights in the Group are still usable. Nothing is saved. The error appears on the check's Metric Group field and names each affected Metric, its Evaluator by ID, and the reason. A gate that has no usable weights at all is refused too, because the Group has no Composite.

To fix it, open the Metric Group, remove or replace the weights for the listed Metrics, and save the gate again. Gates you saved earlier aren't changed, and Scoring Runs still skip unusable weights.

Gate Results​

A gated revision ends in one of these outcomes: the checks passed, a check failed, or the platform couldn't decide. Only a passed revision goes live unless someone overrides a failed result. You can follow the result on the service's Overview tab and in its deployment history.

When the Platform Can't Decide​

A check is undecidable when the platform can't trust its score. This happens, for example, when the Metric Group's scoring run fails, or when the results come from an Evaluator version that the check did not pin. A gate with no checks is also undecidable. If any check is undecidable, the revision doesn't go live, even if the other checks passed. The previous revision keeps serving requests.

An undecidable result is not a quality failure: the platform couldn't judge the revision. After you fix the cause, for example a failed scoring run, you can retry the result. For the steps, see Follow a Gate Result.

Overriding a Failed Gate Result​

Sometimes a revision must ship although its gate failed, for example when a hotfix can't wait, a judge gives unreliable scores, or the threshold is set too high. An override puts the held revision of a failed Gate Result live and keeps a record of the decision.

Overriding works like this:

  • Who can override: Anyone with write permission for services in the workspace.
  • Reason required: The person overriding must give a reason. A blank reason is rejected. The reason can be up to 2,000 characters.
  • Which results qualify: Only a failed Gate Result, including a result the platform couldn't decide. A result that passed, or one that was already overridden, can't be overridden.
  • Audit record: The Gate Result changes to the Overridden outcome and keeps who overrode it, when, and why, next to the verdict it replaced. The activation of the revision is recorded under the person who overrode the gate.

The override is refused, and nothing is recorded, in these cases:

  • Another gate run of the same service is still in progress. Wait for it to finish, then try again.
  • The service was deactivated since the revision was held.
  • The service moved on to another revision since the revision was held. Override the Gate Result of the current held revision instead, if there is one.

You can override a failed Gate Result in the platform UI or through the platform API. For the UI steps, see Follow a Gate Result.

Rollbacks​

The gate checks revisions that are about to go live for the first time. It doesn't hold a rollback, so you can recover quickly when something goes wrong. A rollback is re-activating a revision that was already the service's active revision at some point. This applies whether or not that revision served a query, and even if it once failed a gate run but later went live.

A revision that never went live is not a rollback. For example, a revision you pushed earlier but never activated is still held by the gate when you activate it.

A revision that is already serving or still rolling out after passing the gate is never held again.