Skip to main content
For the complete documentation index for agents and LLMs, see llms.txt.

Trace with Built-In Traces

You can trace your pipelines through built-in traces in the UI. Traces give you full observability into your deployed pipelines and agents without adding a separate observability tool to your stack: everything stays on the same infrastructure your pipelines already run on.

Use deepset MCP Server

For more advanced use cases, use the deepset MCP server to interact with traces and other resources. For details, see Using deepset MCP Server.

Prerequisites​

Traces require no setup. As soon as you deploy or redeploy a pipeline, it starts generating traces automatically for every run.

To learn more about traces, see Monitor Pipeline Performance and Trace Your Pipelines.

info

Traces are collected for queries run on or after 25 June 2026. Before that date, traces are not available. To collect traces for pipelines deployed before that date, undeploy the pipeline and deploy it again.

What a Trace Captures​

A trace is the complete execution record of one pipeline run, from the input that triggered it to the response it returned. Each trace captures:

  • Spans: A hierarchical tree of every component that ran, with per-step timing, component type, and metadata tags.
  • Token usage: Prompt, completion, reasoning, and cache tokens, normalized across providers and attributed to the model that used them. This lets you compare token consumption across models even when an agent uses more than one.
  • Inputs and outputs: The data flowing into and out of every component.
  • Logs: Every structured log entry from the run, with its level and source.
  • Failures: The exception type, message, and full stack trace, with the root-cause component flagged so you know where the run failed and whether it's a configuration error you need to fix, or an external provider or infrastructure error that just needs a rerun.
  • Metadata: Status, total latency, timestamps, the source that triggered the run (Playground, Builder, API, or a shared prototype), and the pipeline version that served it.

View Traces​

  1. Log in to Haystack Enterprise Platform.
  2. Click Pipelines, find the pipelines whose traces you want to see, and click Analytics next to it.
  3. Open Traces. You can see a list of all traces for the pipeline, each with its input, status, source, latency, total tokens, and date and time. Each trace represents the full journey of a single request as it moves through the pipeline.
tip

Filter traces by status, duration, source, or date range to narrow down the list to a release window or a specific day. Sort by latency to find the slowest runs, or search by trace ID or input text to find a specific run, for example, one a customer reported.

View Trace Details​

  1. On the Traces page, click a trace to open its details. Use the filters to find the traces you want to check.
  2. Click any trace to open its details:
    • Use the top of the page to check the key information about the trace, including the pipeline version, status, source, root cause components, and more.
    • Use the Spans section to see a hierarchical tree of all spans in the trace. You can search spans by name and filter by status. The Waterfall panel shows a timeline of all spans in the trace. You can see the start and end time of each span, and the duration of each span. Click a span to select it and see its details. Expand or collapse a span to see its children or parents.
    • Use the Logs section to see the logs related to the selected span or the overall trace.

Drill into any component to see which one drove the most latency or token consumption. When a run fails, the trace names the root-cause component and shows whether the failure was a configuration error you need to fix, or an external provider or infrastructure error that you can typically resolve with a rerun.

Run a Component from a Trace​

If a component failed or produced unexpected results in a recorded trace, you can open it directly in Builder and re-run it with the original trace input. This lets you reproduce a specific run, experiment with different inputs, and debug issues.

  1. In Haystack Enterprise Platform, go to Pipelines and open a pipeline.
  2. Go to Analytics > Traces and open a trace.
  3. In the trace detail view, open the span for the component you want to investigate. If the trace contains a failed component, the failed span is selected automatically.
  4. Click Run in Builder on the span.
    The Run in Builder button on the span
    The Builder opens and the component run widget appears.
  5. Click Fill input from trace to run the component with the input recorded in the trace and click Run Component.
  6. To restore the original trace input at any time, click Fill input from trace in the widget.
  7. To go back to the trace and re-select the component, click Back to trace.
Trace Session Indicator

While you're working in Builder with a trace as context, you're in the inspecting a trace mode. To stop the trace session and return to normal Builder mode, close the inspecting trace indicator.

Investigate Traces with a Coding Agent​

If you're debugging from your IDE, or need to sift through dozens of runs at once, point a coding agent at your traces through the deepset MCP server instead of reviewing them one by one in the UI. The MCP server lets your agent list and read traces, including logs and span tags, so it can look for patterns across many runs at scale. For details, see Using deepset MCP Server.

On the Traces page, next to a trace, click More Actions > View in Search History next to a trace.

You can also navigate between traces and Search History: jump from a trace to its Search History entry, or open the Trace Detail page directly from a query's detailed view in Search History.

Compare Traces​

Every change you make to a pipeline, like a different model or a reworked prompt, might improve your output, but it can also change what the pipeline costs to run and how it behaves. Use Compare to weigh that trade-off instead of guessing at it.

Compare two traces side by side to see exactly what changed: how latency and token consumption moved at each component, and how the answer itself changed, with the two outputs shown side by side and differences highlighted.

For example, if you swap in a larger generation model, Compare shows you what it cost: added latency and tokens at the generation step, while the rest of the pipeline holds steady.

You can also use Compare to catch regressions you didn't intend. Run the same input against a past pipeline version and the current one. If a component slowed down or started returning something different, you'll see it before your users do.

  1. On the Traces page, enable Compare.
  2. Choose up to two traces to compare. Compare works well when you run the same input against two different pipeline versions.
  3. Choose a component from the pipeline run section to compare its inputs and outputs, latency, and token consumption for each trace.

Export Traces as JSON​

  1. Open a trace from the Traces page.
  2. Click More Actions > Export as JSON.

Data Security​

Traces are stored on deepset-owned infrastructure, so trace data is never shipped to an outside observability tool. Access is scoped to each workspace with role-based controls, and data is encrypted at rest. You can export any trace as JSON whenever you need it.