Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Create a ServiceBeta

Beta
This is coming — this page describes a feature that's still being rolled out and may not be available to you yet.

A service runs your pipeline and serves queries. Services decouple the pipeline from the endpoint that serves it. This way, you can continue to iterate on the pipeline in the Builder while keeping a stable and reliable endpoint for external callers. You create and manage services in Gateway.

About This Task​

A service is a stable endpoint that exposes a published pipeline version to your callers. You can set the hosting settings for the service to control how it scales and how long it stays active.

A single service can have one pipeline version published to it. You can publish a new version to a service at any time and the endpoint URL stays the same.

A single pipeline version can be published to multiple services.

To learn more, see Gateway.

Prerequisites​

You must have read and write permissions to services to perform this task. For details, see User Roles and Permissions.

Create a Service​

  1. In the navigation, go to Gateway > Create service.
  2. Give your service a name and, optionally, a description to easily identify it.
  3. Configure the hosting settings for the service:
Hosting settings

Hosting determines the minimum and maximum number of replicas that must be available for the service and idle timeout. Idle timeout is the period of inactivity after which the service becomes inactive to save resources. You can update these settings at any time in the service settings.

info
  • High demand: Choose this option for production pipelines that need high availability and scalability.
  • Low demand: Choose this option for testing and pipelines that are not used frequently.
  • Custom: Configure your own hosting settings.
  1. Click Continue.
  2. Choose the pipeline and the version you want to publish to the service or click Create service without pipeline to publish one later.
  3. If you chose to publish a pipeline version, verify it and add a message with your changes. The message is shown in the service deployment history.
  4. Click Create service. The service is starting. Wait until it's ready to serve queries.

What To Do Next​