Troubleshoot Services
Find the symptom you see, and follow the fix. This page covers problems with starting, publishing to, and connecting a service.
Queries Are Slow or Fail Right After a Start or Switch
If queries are slow, a stream shows only ping events with phase set to cold_start, or queries fail with Too many pending requests for this version right after an activation or a revision switch, the replicas are probably still loading the pipeline.
- Send test queries to warm the service.
- Wait until the queries answer at normal speed.
- Retry your queries.
If queries still fail or stay slow, open a trace to find the slow component. For the steps, see Monitor and Debug a Service. To learn why this happens and how to avoid it, see What Min Replicas Guarantees.
A Publish Is Blocked for a Missing Haystack Version
A service runs on the Haystack version that its pipeline configuration pins. If the pipeline version you publish doesn't pin one, the comparison step shows an error notice and disables Publish and Create service. The platform doesn't pin a version for you. This also applies to new versions of existing services.
To fix it:
- Click Pipeline Settings in the notice, or open the pipeline's Settings from Pipelines.
- Choose a version from the Haystack Version list. This saves a
haystack-aidependency to the pipeline configuration. If you edit the YAML directly, addhaystack-ai==<version>underdependencies. - Publish the pipeline version again. The error notice disappears and Publish or Create service is enabled.
The check covers pipeline versions you publish from Builder. It doesn't cover republishing an externally pushed revision, or creating a service or publishing an initial version from the Gateway services list or the migrate-to-service flow. In those cases the interface doesn't check the pin, and the platform rejects the request when the version has none. Pin the version first. If you publish with the REST API, a revision request without a pinned version returns a 422 error. Add the pin to the config_yaml in your request and retry.
Pushing a Revision with Inline YAML Returns a 422 Error
If you push a revision with inline config_yaml and no source_version_id, the request must set source_type to EXTERNAL_PIPELINE. The default source_type is PLATFORM_PIPELINE, so without it the request fails with a 422 error because the revision isn't linked to a pipeline version.
Choose the fix based on your goal:
- To keep the service linked to a pipeline you built in the platform, publish that pipeline version first and send its version ID as
source_version_id. - To push code that has no platform pipeline, set
source_typetoEXTERNAL_PIPELINE. The revision then isn't linked to a pipeline version.
For the request, see Create Deployment Revision. For a worked request, see Run a Pipeline Larger Than 256 KB with the REST API.
A Revision Activation Is Refused
If activating a revision returns a 409 or 422 error, a gate or a running check is holding it. For each error code and its fix, see Activate a Service Revision with the REST API.
The Slack Connection Shows an Error
Check the symptom you see:
- The Slack row shows an error instead of actions: The platform couldn't load the Slack connection. Reload the page and try again.
- The actions are disabled: You need write permission for services. Ask your organization admin.
- The token is rejected: Use the bot token that starts with
xoxb-, and check that you copied the signing secret from the same Slack app.
For the setup steps, see Connect a Service to Slack.
What To Do Next
- Monitor and Debug a Service to trace queries and check hardware metrics.
- Configure Service Hosting and Cost to change replicas and instance size.
Related Information
Was this page helpful?