Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Create a ServiceBeta

Beta
This is coming — this page describes a feature that's still being rolled out and may not be available to you yet.

A service runs your pipeline and serves queries. Services decouple the pipeline from the endpoint that serves it. This way, you can continue to iterate on the pipeline in the Builder while keeping a stable and reliable endpoint for external callers. You create and manage services in Gateway.

About This Task​

A service is a stable endpoint that exposes a published pipeline version to your callers. You can set the hosting settings for the service to control how it scales and how long it stays active.

A single service can have one pipeline version published to it. You can publish a new version to a service at any time and the endpoint URL stays the same. You can publish a single pipeline version to multiple services.

When you create a service through the API without setting deployment_mode, the service runs in serverless mode. Callers that relied on the earlier managed default now get a serverless service. If you need a managed service, set deployment_mode to MANAGED in the request. For details, see the Create Deployment API reference.

To learn more, see Gateway.

Prerequisites​

You must have read and write permissions to services to perform this task. For details, see User Roles and Permissions.

Create a Service​

  1. In the navigation, go to Gateway > Create service.
  2. Give your service a name and, optionally, a description to easily identify it. The name must be unique in the workspace. Names are case-sensitive, so Support and support are different names.
  3. Optional: Add up to three tags to label the service.
  4. Configure the hosting settings for the service.
  • Select the Scaling preset, which sets how many replicas run.
Hosting settings

Hosting determines

the minimum and maximum number of replicas that must be available for the service and idle timeout. Idle timeout is the period of inactivity after which the service becomes inactive to save resources. You can update these settings at any time in the service settings.

5. Click Continue. 6. Choose the pipeline and the version you want to publish to the service or click Create service without pipeline to publish one later. If your workspace has no pipelines yet, use this option and publish a pipeline later with the CLI or from the platform. 7. If you chose to publish a pipeline version, verify it and add a message with your changes. The message is shown in the service deployment history. 8. Click Create service. The service is starting. Wait until it's ready to serve queries.

Create a Service with the CLI​

If your pipeline is in code, you can create the service and publish your first version in one step with the Haystack Enterprise SDK. Only Haystack pipelines and agents are supported.

  1. In Gateway, on the Create service page, click Have a pipeline in code? Create the service with the CLI instead. If your workspace has no services yet, click Create with the CLI in the empty state.
  2. In the Create a service with the CLI drawer, select With a coding agent or With the CLI.
  3. Copy the prompt or the commands and run them. Replace my-service with your service name and <your_pipeline.py> with the Python file that defines your pipeline or agent.

The first run creates a new service with your pipeline as its first version. The service starts with default hosting settings, which you can change later in the service settings. The command fails if the service name is already taken. To publish an update later, run it again with --no-create instead of --create.

What To Do Next​