Monitor and Debug a ServiceBeta
Find slow or failing runs, see how a query moved through the pipeline, and check whether the service has enough CPU and memory.
Prerequisites
- You must have read permissions to services. For details, see User Roles and Permissions.
- You must have a service with a published pipeline. For details, see Publish a Pipeline to a Service.
Trace and Debug Queries
A trace is the record of one query's run. It shows each component that ran, its inputs and outputs, its duration, and any errors, so you can find where a run slowed down or failed.
- In Haystack Enterprise Platform, click Gateway in the navigation and open the service.
- Click Traces.
- Open a trace to inspect its steps.
Tracing works the same as for pipelines. For details on reading and filtering traces, see Trace with Built-In Traces.
Check Service Metrics
To see how a service performs under load, open its Metrics tab. Use it when a serverless service runs slowly or fails, or before you change its replicas or instance size. The tab is in beta and is available only for serverless services.
- In Haystack Enterprise Platform, click Gateway in the navigation.
- Click the name of the service you want to check.
- Click Metrics.
- Review the CPU usage, memory usage, and replica count charts, grouped by pod.
If the service needs more resources, raise Min replicas or Max replicas, or change its instance size. For steps, see Scale the Service and Control Cost.
Find a Cheaper Configuration with the AI Operator
To find out whether the service can cost less without answering worse, set up an AI Operator for it. The Operator measures the service's cost and answer quality and suggests cheaper configurations that are expected to keep that quality. Suggestions are advisory only: the Operator never applies a change to your service for you.
On the Service page, next to the actions menu, click Set up AI Operator if the service has no Operator yet, or AI Operator to open its existing one. For steps, see Set Up an AI Operator for a Service and Start an Optimization Run.
What To Do Next
- Scale the Service and Control Cost if the metrics show that the service needs more resources.
- Trace with Built-In Traces for more on reading and filtering traces.
Related Information
Was this page helpful?