Configure Service Hosting and CostBeta
Tune how much compute a service gets, how many replicas run, and how long the service stays active. These settings balance responsiveness against cost.
Prerequisites
- You must have read and write permissions to services. For details, see User Roles and Permissions.
- You must have a service with a published pipeline. For details, see Create a Service.
Scale the Service and Control Cost
These settings apply only to this service. If the same pipeline is also deployed on its own, configure that deployment separately in the pipeline deployment settings.
Credits consumed ≈ min replicas × hours deployed. Services on standby or in inactive status consume no credits.
- In Haystack Enterprise Platform, click Gateway in the navigation and open the service.
- Click Settings and go to the Hosting section.
-
For a service on serverless hosting, to give each replica more or less compute, select a size under Instance size. The list offers the sizes your organization has enabled: Small, Medium, Large, or Extra large. Organization admins choose these sizes in Organization Settings. For details, see Managing Compute Settings. If the service already runs on a size your organization has since turned off, that size stays in the list so you can see the current value. Confirm the change when prompted.
The service's replicas restart on the new size. Queries already running finish on the old replicas first. If your organization has no capacity for the size you chose, an error appears and the size stays the same. Contact your Organization Admin.
- Choose a preset, or select Custom to set your own values:
- High demand: For production traffic that needs high availability and scaling.
- Low demand: For testing or services that are used rarely.
- Custom: Set Min replicas, Max replicas, and Idle timeout yourself.
- Confirm your changes.
You can change these settings at any time. To check whether the service needs more resources before you change them, see Check Service Metrics.
To cap how many requests one replica serves at the same time, see Limit Requests per Replica with the REST API.
Use a Fleet Environment or Custom Hosting
To give a service the same hosting as other services, connect it to a fleet environment. It then inherits the environment's size, replicas, and idle timeout instead of using its own. Only services on serverless hosting can use a fleet environment.
- Open the service, click Settings, and go to the Hosting section.
- Choose Environment and pick an environment from the list, or choose Custom to set the service's own Min replicas, Max replicas, and Idle timeout.
- Confirm the restart when prompted.
With Environment, the environment's settings show read-only, with a Manage environments link to the Environments tab where you can change them. Switching between Environment and Custom, or between two environments, restarts the service. If you switch to Custom, the service restarts with its own saved settings.
Services in the same environment share instances, so their requests add up to the environment's load. When you check CPU and memory on the Metrics tab for a service that shares an environment, the charts show usage for every service in that environment, not only this one. Their titles end with "(shared across the environment)".
Only an organization admin can create, rename, or delete fleet environments. To check the load on a shared environment, see View Environment Metrics and Check an Environment's Resource Usage. For details on creating and editing environments, see Manage Fleet Environments.
Keep a Service Active with Scheduled Calls
Turn on scheduled calls to make the service call itself at regular intervals, so it doesn't go idle between real requests. This helps if your service gets sporadic traffic and you still want fast response times. Scheduled calls work only for services on non-serverless hosting.
- Open the service, click Settings, and go to the Scheduled Calls section.
- Turn on Enabled.
- In Every (minutes), set how often the service should be called.
- Optional: In Query, type the text to send with each scheduled call. Leave it empty to dispatch an empty query.
- Optional: In Params (JSON), type extra parameters to send as valid JSON.
- Click Save.
If a schedule already exists, the section shows Last call, which is the time of the most recent scheduled call. Turn off Enabled and save to stop scheduled calls without losing your saved interval, query, and parameters.
What To Do Next
- Monitor and Debug a Service to see how the service behaves under load.
- Manage a Service to deactivate, wake, or delete the service.
Related Information
Was this page helpful?