> For the complete documentation index, see [llms.txt](https://docs.cloud.deepset.ai/llms.txt)

Last modified: 2026-11-04

# Gateway

Gateway is where you publish your pipelines to production, giving your callers a stable endpoint that doesn't change when you update the pipeline behind it.

* * *

## What Is Gateway?

Gateway separates work in progress from what is live in production. You build and iterate in the Builder, and when a pipeline is ready, you publish it through Gateway. Callers connect to what you publish, not to the pipeline itself. This way, the connection stays stable no matter how many times you update or rebuild the pipeline behind it.

Gateway lives under its own section in the navigation, where you can manage everything you've published from your current workspace.

### Permissions

To access Gateway, you need the `services: read` permission. To create, update, or delete services, publish pipeline versions to services, or configure service hosting, you need the `services: write` permission. To view a service's secrets, you also need read access to secrets. To create, edit, or delete them, you need write access to secrets. The `services: write` permission alone isn't enough.

## How Gateway Fits into Your Workflow

Haystack Enterprise Platform follows a build → test → publish journey:

1.  Build: Create and iterate on your pipeline in the Builder.
2.  Test: Try your pipeline in the Playground or Prompt Explorer.
3.  Publish: Use Gateway to pin a pipeline version and expose a stable endpoint.

## Services

Inside Gateway, you publish a pipeline by creating a service. A service is a stable, production-ready endpoint that exposes one specific pipeline version to your callers. Publishing a pipeline to a service ensures that the pipeline's availability and performance stay consistent. You can keep updating the pipeline without interrupting the service.

You can publish one pipeline version to a service at a time. To publish multiple pipelines through Gateway, create one service for each pipeline. You can also publish the same pipeline to multiple services. This is useful if you want different service levels for different use cases, or if you want to publish different pipeline versions at once.

You publish a pipeline to a service from the Builder, where you choose the version to publish and either select an existing service or create a new one.

Callers connect to a chat or generative service through either the native Platform API or the OpenAI-compatible Completions API. For steps, see [Call the Service from Your App](https://docs.cloud.deepset.ai/docs/connect-to-a-published-service.md#get-the-service-endpoint).

You can also create a service without a pipeline and publish a local Haystack pipeline or agent to it later with the CLI or a coding agent. For details, see [Publish a Pipeline to a Service](https://docs.cloud.deepset.ai/docs/publish-pipeline-to-service.md).

You can hold new revisions of a service until they pass a Deployment Gate. For details, see [Deployment Gates](https://docs.cloud.deepset.ai/docs/deployment-gates.md).

### Main Benefits

-   Stable production endpoints: Callers connect to a service, not a pipeline. The pipeline can keep changing underneath without affecting what is live.
-   Safe iteration: Experiment freely on your pipeline without touching what your users depend on.
-   Version control with intent: Promote a pipeline version to a service when it is ready, instead of relying on auto-generated history.
-   One pipeline, multiple services: Dev, staging, and production each independently managed from the same pipeline.

### Publishing to a Service vs. Deploying a Pipeline

You can make a pipeline available in two ways: by publishing it to a service, or by deploying it without a service:

-   Deploying a pipeline makes it available for queries directly. It's best for quick testing and internal use, where endpoint stability is not a priority. When you update the pipeline and want to deploy a new version, the endpoint URL will change. Deploying a pipeline is required to test it in the Playground.
    
-   (Recommended) Publishing to a service assigns a specific pipeline version to a stable endpoint. The endpoint doesn't change when you update the pipeline. This is the recommended approach for production integrations, where your callers need a consistent endpoint they can rely on.
    

When building a pipeline other systems will connect to, use a service.

### Service Metrics and Traces

Clicking a service in Gateway opens the Service Overview page, where you can review the service's performance and history, connect to it, and update its settings. For what the metrics and traces show, see [Service Metrics and Traces](https://docs.cloud.deepset.ai/docs/service-metrics-and-traces.md).

### Hosting Settings

Each service has its own hosting settings that define how scalable and reliable it is, such as the minimum and maximum number of replicas, idle timeout, and maximum concurrency per replica. You review and update them on the **Settings** tab of the service. Hosting settings for a service are separate from the pipeline's hosting settings. When you publish a pipeline to a service, it uses the service's hosting settings.

A service can also inherit its hosting settings from a shared fleet environment.

For what each setting does and our recommended values, see [Service Hosting](https://docs.cloud.deepset.ai/docs/service-hosting.md).

## Service Secrets

A service can have its own secrets, separate from workspace and organization secrets. If a service secret has the same name as a workspace or organization secret, the service uses its own. This lets you give one service its own credentials, for example a production API key, without changing what other services in the workspace use.

You add and manage service secrets on the **Secrets** tab of the service details page. For steps, see [Add Secrets to a Service](https://docs.cloud.deepset.ai/docs/add-secrets-to-a-service.md).

## Service Status

Possible statuses of a service are:

-   **Active** — the service is deployed and ready to handle requests.
-   **Standby** — the service is idle to save resources. It reactivates automatically when a request arrives, or you can reactivate it manually. For details, see [Manage a Service](https://docs.cloud.deepset.ai/docs/deactivate-or-delete-a-service.md#deactivate-a-service-to-save-resources).
-   **Deploying** — the service is being deployed. On a first publish, nothing is serving yet. When you publish a new version, the version already serving keeps handling traffic until the rollout finishes. If the rollout fails, the last working version keeps serving, so callers don't notice. To see which pipeline version is rolling out, open the pipeline's versions list in Builder. The version shows a *Publishing to 1 service* tag. For details, see [Manage Pipeline Versions](https://docs.cloud.deepset.ai/docs/manage-pipeline-versions.md).
-   **Failed** — the service could not be deployed or encountered a fatal error.
-   **Inactive** — the service has been deactivated.

You can deactivate a service from the Service Details page. To activate it again, use **Activate service** on that page.

## AI Operator

To lower what a service costs without losing answer quality, attach an AI Operator to it. An AI Operator is an optimization goal for one service: it measures what the service costs and how well it answers, and then suggests cheaper configurations that keep the quality you need. For details, see [AI Operator](https://docs.cloud.deepset.ai/docs/ai-operator.md). To add one, see [Set Up an AI Operator for a Service](https://docs.cloud.deepset.ai/docs/set-up-ai-operator.md).

## Durable Runs

A durable run is an alternative way to submit a request to a service. You get a receipt immediately, and the pipeline keeps working even if you close the browser or lose your connection. Use durable runs for long-running pipelines. For details, see [Durable Runs](https://docs.cloud.deepset.ai/docs/durable-runs.md).

## Related Information

-   [Publish a Pipeline to a Service](https://docs.cloud.deepset.ai/docs/publish-pipeline-to-service.md)
-   [Service Hosting](https://docs.cloud.deepset.ai/docs/service-hosting.md)
-   [Service Metrics and Traces](https://docs.cloud.deepset.ai/docs/service-metrics-and-traces.md)
-   [Connect to a Published Service](https://docs.cloud.deepset.ai/docs/connect-to-a-published-service.md)

-   [Pipeline Evaluation](https://docs.cloud.deepset.ai/docs/pipeline-evaluation.md)
-   [AI Operator](https://docs.cloud.deepset.ai/docs/ai-operator.md)
